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Why  GAO  Did  This  Study 

This  is  GAO’s  annual  assessment  of 
DOD’s  major  weapon  system 
acquisitions,  an  area  on  GAO’s  high- 
risk  list.  DOD  and  Congress  have 
taken  steps  to  improve  the  acquisition 
of  weapon  systems.  Still  some 
programs  continue  to  experience  cost 
and  schedule  overruns.  GAO 
continues  to  emphasize  the  need  for 
DOD  and  Congress  to  hold  programs 
accountable  by  ensuring  that  they 
attain  the  required  knowledge  at  key 
decision  points  before  committing 
resources  to  development.  Given 
continuing  budgetary  pressures,  DOD 
cannot  afford  to  miss  opportunities  to 
address  program  inefficiencies. 

The  joint  explanatory  statement  to  the 
DOD  Appropriations  Act,  2009  includes 
a  provision  for  GAO  to  annually  review 
DOD’s  portfolio  of  weapon  systems. 
This  report  includes  observations  on 
(1)  the  cost  and  schedule  performance 
of  DOD’s  2016  portfolio  of  78  major 
defense  acquisition  programs,  (2) 
program  implementation  of  key 
acquisition  reform  initiatives,  and  (3) 
the  knowledge  attained  at  key  decision 
points  for  54  weapon  programs  in 
development  or  early  production. 

GAO  analyzed  cost,  schedule,  and 
quantity  data  from  DOD’s  Selected 
Acquisition  Reports  from  2010,  2014, 
and  2015.  GAO  also  collected  program 
office  data  through  two  questionnaires 
on  technology,  design,  and 
manufacturing  knowledge;  the  use  of 
knowledge-based  acquisition  practices; 
and  the  implementation  of  acquisition 
reforms  and  initiatives. 

In  commenting  on  a  draft  of  this  report, 
DOD  agreed  with  GAO's  findings  and 
stated  that  the  results  validate  its  focus 
on  continuous  program  improvements. 

View  GAO-1 7-333SP.  For  more  information, 
contact  Michael  J.  Sullivan  at  (202)  512-4841 
or  sullivanm@gao.gov 


What  GAO  Found 

Since  GAO’s  201 6  assessment,  the  number  of  programs  in  the  Department  of 
Defense’s  (DOD)  portfolio  of  major  defense  acquisitions  decreased  from  79  to 
78,  while  DOD’s  planned  investment  over  the  life  of  these  programs  increased  by 
$9.4  billion  to  $1 .46  trillion.  GAO  observed  mixed  performance  in  the  portfolio 
this  year.  For  example,  although  the  current  portfolio  has  incurred  $484  billion  in 
total  cost  growth,  $476  billion  of  this  occurred  5  or  more  years  ago,  indicating 
recent  performance  has  improved.  Yet,  60  percent  of  the  total  cost  growth 
occurred  after  programs  entered  production,  when  costs  should  be  more  stable. 
However,  the  portfolio  increased  its  buying  power  by  $10.7  billion — meaning 
DOD  is  able  to  buy  more  goods  or  services  for  the  same  level  of  funding.  This 
gain  resulted  from  some  programs  finding  procurement  efficiencies  that  more 
than  offset  inefficiencies  in  other  programs.  The  19  newest  programs  decreased 
their  costs  by  a  combined  $3.4  billion  over  the  past  year. 

Implementation  of  key  reform  initiatives  GAO  analyzed  for  the  45  current  and  9 
future  programs  it  assessed  this  year  was  similar  or  slightly  less  as  compared  to 
its  201 6  assessment.  These  initiatives  address  program  and  portfolio 
affordability,  cost  growth  controls,  and  competition  use.  For  example,  42 
programs  this  year  reported  conducting  “should-cost”  analyses,  which  are 
designed  to  reduce  programs’  costs  by  identifying  and  eliminating  inefficiencies. 
Programs  can  take  a  variety  of  actions,  such  as  reducing  overhead,  to  do  so. 
Programs  implementing  “should-cost”  analyses  reported  realizing  $23.6  billion  in 
savings,  although  they  could  not  identify  to  GAO  exactly  where  $1 1 .0  billion  of 
these  savings  were  transferred  within  DOD.  Also,  a  similar  number  of  programs 
reported  establishing  affordability  constraints,  which  programs  use  to  set 
priorities  and  inform  what  they  can  and  cannot  afford.  GAO  found  that  current 
programs  with  established  affordability  constraints  had  a  lower  average  amount 
of  cost  growth  from  initial  estimates  compared  to  programs  without  a  constraint. 
However,  DOD’s  implementation  of  another  key  reform  initiative — the  fostering  of 
competitive  environments  in  acquisition — is  stagnant.  Compared  to  GAO’s  2016 
assessment,  more  programs  this  year  reported  having  no  plans  for  competition 
before  or  after  development  start.  Lastly,  31  current  programs  reported  that  they 
are  scheduled  to  complete  the  evaluation  of  their  potential  cyber  vulnerabilities 
by  201 9,  required  by  the  201 6  fiscal  year  National  Defense  Authorization  Act. 

Most  of  the  programs  GAO  assessed  this  year  are  not  yet  fully  following  a 
knowledge-based  acquisition  approach,  as  GAO  has  previously  recommended. 
This  held  true  for  the  four  programs  that  recently  entered  system  development  as 
only  one  completed  all  of  the  applicable  GAO  criteria  for  a  best  practices 
approach.  Three  of  the  four  implemented  some  knowledge-based  practices, 
such  as  completing  a  system-level  preliminary  design  review  before 
development  start.  Meanwhile,  other  practices,  such  as  fully  maturing 
technologies  prior  to  system  development  start  and  completing  systems 
engineering  reviews,  were  not  fully  implemented.  As  a  result,  these  programs  will 
carry  unwanted  risk  into  subsequent  phases  of  acquisition  that  could  result  in 
cost  growth  or  schedule  delays.  In  addition,  a  number  of  programs  are 
concurrently  conducting  software  and  hardware  development  during  production, 
further  exposing  programs  to  undue  cost  and  schedule  risk. 
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U.S.  GOVERNMENT  ACCOUNTABILITY  OFFICE 


441  G  St.  N.W. 
Washington,  DC  20548 


Comptroller  General 
of  the  United  States 


March  30,  2017 
Congressional  Committees 

I  am  pleased  to  present  GAO’s  15th  annual  assessment  of  the 
Department  of  Defense’s  (DOD)  major  defense  acquisition  programs. 

This  report  offers  observations  on  the  performance  of  DOD’s  current 
$1.46  trillion  portfolio  of  78  programs.1  These  78  programs  will  require 
just  over  one  quarter  of  all  DOD’s  development  and  procurement  funding 
over  the  next  5  years.  This  significant  financial  investment  demands  keen 
oversight  and  continued  implementation  of  key  legislative  and  policy 
reforms  as  well  as  knowledge-based  best  practices  developed  by  GAO.2 
Over  the  past  several  years,  we  have  found  that  many  DOD  programs 
have  not  uniformly  implemented  these  reforms  or  best  practices. 

Our  current  assessment  shows  that  the  progress  DOD  has  made  since 
2010  to  decrease  the  amount  of  cost  growth  in  its  portfolio  has  now 
flattened  out.  When  compared  to  last  year’s  portfolio,  the  2016  portfolio 
includes  one  fewer  program  but  is  estimated  to  cost  nearly  $9.4  billion 
more,  as  a  majority  of  programs  experienced  cost  increases  over  the  past 
year.  We  also  found  that  programs  in  the  2016  portfolio  incurred  an 
average  delay  in  delivering  capability  of  almost  2  months.  More 
encouraging,  we  found  that  programs  that  started  development  after  the 
implementation  of  the  Weapon  Systems  Acquisition  Reform  Act  of  2009 
(WSARA)  and  DOD’s  “Better  Buying  Power”  initiatives  began  in  2010  saw 
cost  decreases  over  the  past  2  years,  indicating  that  recent  reforms  may 
be  having  a  positive  effect,  though  the  decrease  was  not  significant 
enough  to  avert  an  overall  cost  increase  in  the  portfolio. 


^ur  assessment  of  DOD’s  portfolio  does  not  include  the  cost  of  the  Ballistic  Missile 
Defense  System  (BMDS),  as  the  program  and  its  elements  lack  acquisition  program 
baselines  needed  to  support  our  assessment  of  cost  and  schedule  change.  For  more 
information,  on  BMDS  and  its  elements  see  GAO,  Missile  Defense:  Ballistic  Missile 
Defense  System  Testing  Delays  Affect  Delivery  of  Capabilities,  GAO-1 6-339R 
(Washington,  D.C.:  Apr.  28,  2016).  10  U.S.C.  §  225  requires  MDA  to  establish  and 
maintain  an  acquisition  baseline  for  certain  elements  of  the  BMDS,  but  these  baselines 
are  not  the  same  as  the  acquisition  program  baselines  developed  pursuant  to  10  U.S.C.  § 
2435  and  DOD  acquisition  policies.  For  example,  they  do  not  include  service-funded 
operations  and  sustainment  costs  needed  to  support  GAO’s  assessment  of  cost  and 
schedule  change. 

2See,  e.g.,  Weapon  Systems  Acquisition  Reform  Act  of  2009,  Pub.  L.  No.  1 1 1  -23. 
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Nonetheless,  we  found  that  new  programs  continue  to  implement  only 
some  of  the  knowledge-based  best  practices  that  are  necessary  at 
program  start  to  provide  sufficient  knowledge  as  to  whether  a  program  is 
capable  of  meeting  its  performance  requirements  while  also  meeting  its 
cost  and  schedule  commitments.  It  is  at  program  start  that  decision 
makers  have  the  most  leverage  to  ensure  that  the  program  exhibits 
desirable  principles — namely,  a  match  between  its  resources  and 
requirements — before  funding  and  approving  the  program.  A  program’s 
technology  maturity  provides  a  key  indicator  of  whether  this  match  has 
been  achieved.  Our  analysis  also  highlights  other  key  points  in  the 
acquisition  cycle  where  knowledge-based  best  practices  should  be  better 
applied.  For  example,  programs  often  hold  major  design  reviews  and 
enter  production  prior  to  demonstrating  the  product’s  design  is  stable  and 
can  be  manufactured  within  cost,  schedule,  and  quality  targets.  In 
addition,  we  also  observed  that  nearly  half  the  programs  in  the  current 
portfolio  fielded  or  planned  to  field  initial  capabilities  prior  to  fully 
demonstrating  those  capabilities  through  operational  test  and  evaluation. 
Programs  doing  this  risk  fielding  systems  that  are  later  found  to  be 
ineffective  or  unsuitable  for  missions. 

Weapon  system  acquisitions  remain  an  area  on  GAO’s  high-risk  list.3 
Continued  efforts  by  DOD  to  mitigate  these  risks  through  adherence  to 
best  practices  and  effective  program  management  are  essential. 


Gene  L.  Dodaro 
Comptroller  General 
of  the  United  States 


3GAO,  High-Risk  Series:  Progress  on  Many  High-Risk  Areas,  While  Substantial  Efforts 
Needed  on  Others,  GAO-17-317  (Washington,  D.C.:  Feb.  15,  2017). 
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GAO 


U.S.  GOVERNMENT  ACCOUNTABILITY  OFFICE 


441  G  St.  N.W. 
Washington,  DC  20548 


March  30,  2017 
Congressional  Committees 

In  response  to  the  joint  explanatory  statement  accompanying  the 
Department  of  Defense  Appropriations  Act,  2009,  this  report  provides 
insight  into  the  department’s  $1 .46  trillion  portfolio  of  major  weapon 
programs.1  This  report  also  includes  information  related  to  small  business 
participation  pursuant  to  a  mandate  in  a  report  for  the  National  Defense 
Authorization  Act  for  Fiscal  Year  201 3. 2  Additionally,  this  report’s 
assessment  of  the  Navy’s  Presidential  Helicopter  Program  constitutes 
GAO’s  response  to  the  annual  reporting  requirement  for  2017  required  by 
the  National  Defense  Authorization  Act  for  Fiscal  Year  201 4. 3 

This  report  includes  observations  on  (1)  the  cost  and  schedule 
performance  of  DOD’s  2016  portfolio  of  78  major  defense  acquisition 
programs,  (2)  program  implementation  of  key  acquisition  reform 
initiatives,  and  (3)  the  knowledge  attained  at  key  decision  points  in  the 
acquisition  process  for  54  current  and  planned  weapon  programs.4 

Our  observations  in  this  report  are  based  on  three  sets  of  programs: 

•  We  assessed  78  major  defense  acquisition  programs  in  DOD’s  2016 
portfolio  for  our  analysis  of  cost  and  schedule  performance.  We 
obtained  cost,  schedule,  and  quantity  data  from  DOD’s  December 


^ee  Explanatory  Statement,  154  Cong.  Rec.  H  9427,  9526  (daily  ed.,  Sept.  24,  2008),  to 
the  Department  of  Defense  Appropriations  Act,  2009,  contained  in  Division  C  of  the 
Consolidated  Security,  Disaster  Assistance,  and  Continuing  Appropriations  Act,  2009, 
Pub.  L.  No.  110-329  (2008). 

2H.R.  Rep.  No.  112-479,  at  284  (2012).  The  National  Defense  Authorization  Act  for  Fiscal 
Year  2013,  Pub.  L.  No.1 12-239. 

3The  National  Defense  Authorization  Act  for  Fiscal  Year  2014,  Pub.  L.  No.1 13-66,  §  252 
(2013). 

4Major  defense  acquisition  programs  (MDAP)  are  those  identified  by  DOD  or  with  a  dollar 
value  for  all  increments  estimated  to  require  eventual  total  expenditure  for  research, 
development,  test,  and  evaluation  of  more  than  $480  million,  or  for  procurement  of  more 
than  $2.79  billion,  in  fiscal  year  2014  constant  dollars.  DOD  has  a  list  of  programs 
designated  as  future  major  defense  acquisition  programs.  These  programs  have  not 
formally  been  designated  as  MDAPs;  however,  DOD  plans  for  these  programs  to  enter 
system  development,  or  bypass  development  and  begin  production,  at  which  point  they 
will  likely  be  designated  as  MDAPs.  We  refer  to  these  programs  as  future  or  planned 
major  defense  acquisition  programs  throughout  this  report. 
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2015  Selected  Acquisition  Reports  (SAR),  which  detail  initial  cost, 
schedule,  and  performance  baselines  as  well  as  changes  to  baselines 
over  the  past  year,  and  from  the  Defense  Acquisition  Management 
Information  Retrieval  Purview  system,  a  DOD  repository  for  program 
data.  We  assessed  the  reliability  of  the  data  by  analyzing  it  and 
reviewing  the  steps  officials  take  to  ensure  the  system’s  data 
reliability,  and  determined  that  the  data  were  sufficiently  reliable  for 
the  purposes  of  our  report. 

•  We  assessed  45  major  defense  acquisition  programs  currently 
between  the  start  of  development  and  the  early  stages  of  production 
for  knowledge  attained  at  key  decision  points  and  their 
implementation  of  acquisition  reforms.  We  obtained  information  on  the 
extent  to  which  the  programs  follow  knowledge-based  practices — 
identified  by  our  past  work  included  in  the  Related  GAO  Products 
section  of  this  report — for  technology  maturity,  design  stability,  and 
production  maturity  using  two  data-collection  instruments.  One  was  a 
questionnaire  on  issues  such  as  systems  engineering,  design 
drawings,  manufacturing  planning  and  execution,  and  the 
implementation  of  specific  acquisition  reforms.  The  other  was  a  data- 
collection  instrument  for  determining  schedule  dates,  critical 
technology  levels,  and  other  information.  We  received  questionnaire 
responses  from  all  45  current  programs  from  August  through  October 
2016. 

•  We  also  assessed  9  future  major  defense  acquisition  programs  not 
yet  in  the  portfolio  in  order  to  gain  additional  insights  into  knowledge 
attained  before  the  start  of  development  and  plans  for  implementation 
of  key  acquisition  reform  initiatives.  We  submitted  a  questionnaire  to 
program  offices  to  collect  information  on  issues  such  as  program 
schedule  events,  costs,  and  numerous  acquisition  reforms,  and 
received  responses  from  all  9  future  programs  from  August  through 
October  2016. 

We  present  individual  assessments  of  54  programs — 45  current  and  9 

future  weapon  programs.5  These  assessments  include  major  defense 

acquisition  programs  currently  in  development  or  early  production  as  well 


5ln  addition  to  the  54  program  assessments,  we  separately  assessed  the  Navy's  Frigate 
initiative,  which  is  currently  part  of  the  Navy’s  Littoral  Combat  Ship  program.  We  assessed 
the  P-8A  Poseidon  Multi-Mission  Maritime  Aircraft  Increment  3  as  part  of  the  9  future 
major  defense  acquisition  programs  not  yet  in  the  portfolio.  During  the  course  of  our 
review,  we  learned  that  DOD  no  longer  plans  to  manage  Increment  3  separately  from  the 
existing  P-8A  program.  We  will  reflect  this  change  in  future  assessments  of  the  P-8A 
program. 
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as  future  programs.  Appendix  I  contains  detailed  information  on  our 
scope  and  methodology. 

We  conducted  this  performance  audit  from  April  2016  to  March  2017  in 
accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 


Background 


DOD  acquires  new  weapons  for  its  warfighters  through  a  management 
process  known  as  the  Defense  Acquisition  System.6  This  system 
generally  requires  defense  acquisition  programs  to  proceed  through 
phases  of  development,  which  include  a  series  of  milestone  reviews  and 
other  key  decision  points  that  may  authorize  entry  into  a  new  phase  of 
acquisition.  Once  a  decision  has  been  made  regarding  the  concept  of  the 
product  to  be  acquired,  the  program  may  enter  the  initial  phase  of 
technology  development,  known  as  Technology  Maturation  and  Risk 
Reduction.  The  purpose  of  this  phase  is  to  reduce  technology, 
engineering,  integration,  and  life-cycle  cost  risk  to  the  point  that  a 
decision  to  contract  for  system  development  can  be  made  with 
confidence  in  successful  program  execution  for  development,  production, 
and  sustainment.  After  that,  the  engineering  and  manufacturing 
development  phase  is  to  develop,  build,  and  test  a  product  to  verify  that 
all  operational  and  derived  requirements  have  been  met,  and  to  support 
production  or  deployment  decisions.  In  this  report,  we  refer  to  DOD’s 
engineering  and  manufacturing  development  phase  as  system 
development.  The  final  phase,  before  sustainment,  is  production  and 
deployment,  the  purpose  of  which  is  to  produce  and  deliver  requirements- 
compliant  products  to  receiving  military  organizations.  In  the  production 
phase,  the  product  is  produced  and  fielded  for  use  by  operational  units. 
Low-rate  initial  production,  limited  deployment,  operational  test  and 
evaluation,  and  full  rate  production  all  occur  in  this  phase.  These  various 
phases  allow  DOD  to  oversee  and  manage  programs’  development 


department  of  Defense  Directive  5000. 01,  The  Defense  Acquisition  System  (Nov.  2007); 
Department  of  Defense  Instruction  5000.02,  Operation  of  the  Defense  Acquisition  System 
(Jan.  2015)  (“DOD  Instruction  5000.02”).  An  update  to  this  instruction  was  released  in 
January  2017,  after  we  completed  much  of  our  analyses. 
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strategies  and  activities.  Figure  1  illustrates  the  key  milestones  and 
reviews  associated  with  the  Defense  Acquisition  System. 


Figure  1 :  Defense  Acquisition  System 


Development  start  Production  start 

DOD  acquisition  milestones  (Milestone  B)  (Milestone  C) 


- System  development - 1 

Technology 

Development 

Integration 

Demonstration 

Production 

Source:  GAO  analysis  of  Department  of  Defense  (DOD)  Instruction  5000.02.  |  GAO-1 7-333SP 


Each  year,  DOD  releases  Selected  Acquisition  Reports  (SAR)  on  the 
status  of  its  major  defense  acquisition  programs,  including  a 
comprehensive  annual  SAR  that  is  generally  submitted  in  conjunction 
with  the  submission  of  the  President’s  Budget.7  The  cost  estimates 
provided  in  the  SARs  include  four  types  of  funding:  research  and 
development,  procurement,  military  construction,  and  acquisition-related 
operations  and  maintenance.  These  costs  reflect  actual  costs  to  date  as 
well  as  future  anticipated  costs.  We  use  the  SAR  data  to  analyze  and 
report  on  the  progress  of  all  of  the  current  major  defense  acquisition 
programs,  which  we  refer  to  as  the  current  DOD  portfolio.  The  DOD 
Instruction  5000.02  is  implementing  policy  that  describes  the  steps  and 
procedures  that  programs  generally  follow  as  they  proceed  through  the 
defense  acquisition  system.  Since  2013,  this  instruction  has  incorporated 
certain  2010  acquisition  reform  initiatives. 

DOD  and  Congress  have  previously  addressed  some  of  the  challenges 
and  problems  in  the  defense  acquisition  system.  Over  the  years,  these 
entities  have  explored  ways  to  improve  acquisition  outcomes.  Most 
notably,  the  Weapon  Systems  Acquisition  Reform  Act  of  2009  (WSARA) 
aimed  to  improve  the  organization  and  procedures  of  DOD  for  the 
acquisition  of  major  weapon  systems.8  Among  other  things,  WSARA 
revised  the  certifications  that  programs  were  expected  to  complete  before 


7SAR  reporting  is  statutorily  required.  10  U.S.C.  §  2432. 

8Pub.  L.  No.  111-23. 
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being  approved  for  system  development  start.9  Programs  are  currently 
required  to  make  certain  determinations  and  certifications  that  they  have 
met  the  following  requirements,  among  others,  prior  to  entering  system 
development: 

•  Appropriate  trade-offs  among  cost,  schedule,  and  performance 
objectives  have  been  made  to  ensure  the  program  is  affordable; 

•  A  preliminary  design  review  and  formal  post-preliminary  design  review 
assessment  have  been  conducted  and,  on  the  basis  of  such 
assessment,  the  program  demonstrates  a  high  likelihood  of 
accomplishing  its  intended  mission;  and 

•  Technology  has  been  demonstrated  in  a  relevant  environment  on  the 
basis  of  an  independent  review  and  assessment  by  the  Assistant 
Secretary  of  Defense  for  Research  and  Engineering. 

Since  WSARA  was  implemented  in  late  2009,  some  of  its  requirements 
have  been  revised  or  repealed.  The  National  Defense  Authorization  Act 
for  Fiscal  Year  2016  repealed  the  requirement  for  programs  to  conduct 
competitive  prototyping  prior  to  starting  system  development.10  Now, 
acquisition  strategies  are  to  include  competitive  prototyping  before 
starting  system  development,  to  the  maximum  extent  practicable  and 
consistent  with  the  economical  use  of  available  financial  resources,  rather 
than  required. 

In  2010,  DOD  started  its  own  acquisition  reform  initiatives  in  its  “Better 
Buying  Power”  memoranda.  These  reforms  include  requiring  programs  to 
conduct  affordability  and  “should-cost”  analyses,  among  other  things, 
which  can  result  in  placing  cost  constraints  on  programs  and  encouraging 
programs  to  find  cost  improvements  during  program  execution.  In 
addition,  one  of  the  initiatives  in  DOD’s  “Better  Buying  Power”  memos  is 
aimed  at  implementing  practices  and  policies  designed  to  improve  the 
productivity  of  DOD  and  its  industrial  base.  Many  of  the  WSARA 
requirements  and  DOD  initiatives  have  been  incorporated  into  the  DOD 
Instruction  5000.02.  These  and  other  reforms  address  sound 
management  practices,  such  as  realistic  cost  estimating,  prototyping,  and 
systems  engineering.  However,  we  previously  found  that  the  results  of 
these  reforms  and  initiatives  might  be  limited  if  programs  do  not  follow  the 


9This  requirement  is  codified  as  amended  at  10  U.S.C.  §  2366b. 

10Pub.  L.  No.  114-92,  §  822(b). 
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knowledge-based  best  practices  GAO  has  identified  for  product 
development.11 

Observations  on  the 
Cost  and  Schedule 
Performance  of 

DOD’s  2016  Major 
Defense  Acquisition 
Program  Portfolio 

Since  our  last  report  in  2016,  our  analysis  shows  that  DOD’s  total  planned 
investment  in  major  defense  acquisition  programs  increased  by  about 
$9.4  billion  from  $1 .45  trillion  to  $1 .46  trillion,  whereas  the  number  of 
programs  decreased  from  79  to  78.  The  cost  increase  represents  a 
flattening  to  a  trend  of  total  acquisition  cost  decreases  we  observed  each 
year  from  2010  to  2015.  We  attribute  this  aggregate  cost  increase  to  cost 
growth  affecting  a  majority  of  individual  DOD  programs  over  the  past 
year,  and,  in  particular,  significant  cost  increases  in  a  few  large 
shipbuilding  programs. 

Our  analysis  also  shows  that  the  portfolio  has  experienced  cost  growth 
totaling  over  $484  billion  since  programs  established  their  first  full 
estimates;  60  percent  of  the  cost  growth  occurred  after  programs  started 
production.  These  significant  post-production  cost  increases — particularly 
within  development  funds — may  indicate  that  programs  start  production 
without  having  demonstrated  that  a  fully  integrated,  capable,  production- 
representative  prototype  will  work  as  intended.  Notably,  $476  billion  of 
this  cost  growth  occurred  in  programs  5  or  more  years  ago.  Since  201 1 , 
the  portfolio’s  cost  has  only  grown  by  $8.6  billion.  Based  on  our  review  of 
DOD  estimates,  the  amount  of  future  funding  needed  to  complete  the 

2016  portfolio  totals  $573.6  billion,  which  is  a  decrease  from  the  2015 
portfolio  and  is  the  lowest  amount  in  over  a  decade.  The  decreased 
amount  of  future  funding  required  indicates  that  more  of  the  total  cost  of 
the  portfolio  has  been  spent.12  Of  the  $573.6  billion,  $546  billion  is 
planned  for  procurement  and  $27.6  billion,  or  5  percent  of  the  total,  is 
planned  for  development.  Over  the  past  2  years,  the  portfolio  has 
experienced  a  buying  power  gain.  Further,  the  current  portfolio’s  average 
delay  in  delivering  capability  increased  by  almost  2  months  over  the  past 
year,  yet  49  percent  of  programs  in  the  2016  portfolio  intend  to  declare,  or 
have  declared,  initial  operational  capability  on  the  basis  of  limited  or,  in  a 
few  cases,  no  operational  testing. 

1 1  GAO,  Defense  Acquisitions:  Assessments  of  Selected  Weapon  Programs, 

GAO-1 6-329SP  (Washington,  D.C.:  Mar.  31,  2016). 

12When  we  refer  to  spent  funding,  we  are  referring  to  funds  that  may  have  been 
expended,  obligated,  and,  or  allotted. 
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Our  analysis  of  DOD’s  201 6  portfolio  allows  us  to  make  the  following  1 1 
observations: 


Cost  and  Schedule  Performance  Observations 

Changes  to  the  portfolio 

1.  The  2016  portfolio  consists  of  78  programs,  down  from  79,  which  will  cost  over 
$1 .46  trillion  to  acquire.  Over  the  past  year,  the  cost  estimates  of  these  programs 
increased  by  $9.4  billion,  or  less  than  1  percent.9 

2.  Development  and  procurement  costs  increased  by  $3.7  and  $5.6  billion 
respectively.  Three  programs — the  Evolved  Expendable  Launch  Vehicle,  the  Next 
Generation  Operational  Control  System,  and  the  Littoral  Combat  Ship — account  for 
$2.2  billion  of  the  aggregate  development  cost  increase.  The  three  largest 
procurement  cost  increases  were  in  shipbuilding  programs,  which  all  increased 
their  procurement  quantities. 

3.  The  amount  of  future  funding  needed  to  complete  the  portfolio’s  planned 
development  activities  and  procure  all  planned  units — $573.6  billion-decreased. b 
Of  this  future  funding,  $546  billion  is  for  procurement  and  $27.6  billion  is  for 
development.  The  remaining  funds  for  development  total  less  than  half  the  amount 
that  remained  10  years  ago.  The  current  portfolio’s  average  delay  in  delivering 
capability  increased  by  almost  2  months  over  the  past  year. 

Factors  that  explain  the  changes 

4.  The  entire  portfolio  has  experienced  over  $484  billion  in  cost  growth  since 
programs  established  their  first  full  estimates.  However,  $476  billion  of  this  growth 
occurred  5  or  more  years  ago.  The  portfolio  realized  60  percent  of  its  cost  growth, 
or  $259  billion,  after  production  start.  These  post-production  cost  increases  may 
indicate  that  programs  start  production  without  having  demonstrated  that  a 
production-representative  prototype  will  work  as  intended.  Programs  started  before 
201 0  make  up  a  majority  of  the  total  cost  growth  since  first  full  estimates. 

5.  The  portfolio  realized  a  buying  power  gain  of  $10.7  billion,  although  more  programs 
overall  lost  buying  power  over  the  past  year.0 

6.  As  measured  against  specific  cost  growth  metrics,  78  percent  of  programs  meet 
the  threshold  for  less  than  10  percent  cost  growth  over  the  past  5  years.  However, 
fewer  programs  meet  the  metrics  for  realizing  less  than  2  percent  cost  growth  over 
1  year  or  for  less  than  1 5  percent  cost  growth  from  initial  estimates. 

Post-Reform  Program  Observations 

7.  Nineteen  programs  started  development  after  the  implementation  of  acquisition 
reforms  in  201 0.  These  programs  decreased  their  costs  by  $3.4  billion  over  the 
past  year — continuing  a  trend  that  began  in  2015 — but  experienced  an  average 
delay  to  delivering  capability  of  over  3  months. 

8.  These  19  programs  represent  about  one-quarter  of  the  programs  in  the  whole 
portfolio,  but  account  for  only  10  percent  of  the  portfolio’s  total  acquisition  cost. 

9.  Over  the  past  year,  total  acquisition  cost  changes  for  these  1 9  programs  show  a 
similar  profile  to  the  cost  changes  affecting  the  remaining  59  programs.  Further,  an 
inordinately  large  program  in  each  group — Joint  Light  Tactical  Vehicle  and  F-35 
Lightning  II — drives  a  majority  of  the  overall  cost  changes. 
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Other  Observations 

1 0.  Forty-nine  percent  of  programs  intend  to  or  have  declared  initial  operating 
capability  on  the  basis  of  limited  or,  in  a  few  cases,  no  operational  testing. 
Specifically,  9  percent  declared  initial  operating  capability  before  beginning  testing, 
and  40  percent  did  or  plan  to  declare  initial  operating  capability  before  testing 
completes.  Programs  declaring  initial  operational  capability  prior  to  completing 
initial  operational  test  and  evaluation  risk  fielding  systems  that  are  later  found  to  be 
ineffective  or  unsuitable  for  missions. 

1 1 .  The  leading  companies  DOD  relies  on  to  develop  and  produce  military  capabilities 
have  consistently  shown  strong  market  performance,  which  indicates  that  investors 
expect  the  performance  of  these  companies  to  be  strong  for  some  time  to  come. 
Stock  prices  for  these  companies  have  increased  significantly  over  time — 
outperforming  other  market  sectors  over  the  past  1  year  and  5  years. 

Source:  GAO  analysis  of  DOD  data.  |  GAO-1 7-333SP 

aAII  dollar  figures  are  in  fiscal  year  201 7  constant  dollars,  unless  otherwise  noted. 

bDetails  on  program  costs  used  for  this  analysis  are  provided  in  appendix  I.  For  more  information  on 
the  portfolio’s  performance  since  their  estimates  1  year  and  5  years  ago,  see  appendix  II. 

“Buying  power  can  be  defined  as  the  amount  of  goods  or  services  that  can  be  purchased  given  a 
specified  level  of  funding. 


Changes  to  the  portfolio  1.  The  201 6  portfolio  consists  of  78  programs,  down  from  79,  which 

will  cost  over  $1.46  trillion  to  acquire.  Over  the  past  year,  the 
cost  estimates  of  these  programs  increased  by  $9.4  billion  or 
less  than  1  percent.  Our  analysis  shows  that  this  cost  increase  is 
primarily  due  to  three  factors: 

•  First,  three  programs  left  the  portfolio — taking  away  $17  billion 
from  the  total  cost — while  two  new  programs  expected  to  cost 
$16.9  billion  in  total  entered  the  portfolio.  Of  the  three  programs 
that  exited  the  portfolio,  the  Joint  Tactical  Networks  and  the  RQ- 
4A/B  Global  Hawk  Unmanned  Aircraft  System  completed  planned 
development  and  procurement.  One  program,  the  Joint  Standoff 
Weapon,  was  terminated.  Two  Army  programs  began  system 
development  and  entered  the  portfolio:  the  Armored  Multi-Purpose 
Vehicle  and  the  Joint  Air-to-Ground  Missile.  The  Joint  Air-to- 
Ground  Missile  program  was  in  the  materiel  solution  and 
technology  development  phase  before  its  system  development 
start  but  the  Armored  Multi-Purpose  Vehicle  program  entered 
development  with  its  system  development  start  in  December 
2014.  Figure  2  shows  the  changes  in  total  cost  and  number  of 
programs  within  DOD’s  portfolio  of  major  weapon  acquisitions 
since  2006. 
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Figure  2:  DOD  Portfolio  Cost  and  Size,  2006-2016 
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Source:  GAO  analysis  of  Department  of  Defense  data.  |  GAO-1 7-333SP 

Note:  DOD  did  not  issue  SARs  in  2009,  which  precludes  us  from  having  the  cost  baseline  information 
necessary  to  include  2009  in  this  analysis. 


•  Second,  two  of  the  portfolio’s  joint  programs  decreased  their  total 
cost  significantly.  The  F-35  Lightning  M’s  total  cost  decreased  by 
$7.6  billion  over  the  past  year  with  no  changes  to  its  planned 
procurement  quantities.  Much  of  these  savings  were  generated  by 
changes  in  estimating  assumptions  for  savings  in  future  lots.  The 
Joint  Light  Tactical  Vehicle  program  decreased  its  total  quantities 
and  the  total  program’s  cost  decreased  by  $3.8  billion  not  due  to 
the  quantity  changes. 

•  Third,  a  majority  of  programs  experienced  cost  increases  over  the 
past  year.  Of  the  78  programs,  46  saw  $26.8  billion  in  cost 
increases  while  32  programs  experienced  $17.5  billion  in  cost 
decreases.  Most  programs’  cost  change  over  the  past  year  were 
between  a  0  to  5  percent  change,  whether  it  was  an  increase  or  a 
decrease.  However,  we  identified  a  handful  of  programs  with 
percent  cost  changes  greater  than  20  percent,  including  the 
Littoral  Combat  Ship  and  the  Remote  Minehunting  System.  Figure 
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3  shows  the  distribution  of  these  cost  changes  across  the  entire 
portfolio. 


Figure  3:  Distribution  of  the  1-year  Change  in  Total  Acquisition  Cost  for  the  2016  Portfolio 


<-25%  -25-20%  -20-15%  -15-10%  -10-5%  -5-0%  0-5%  5-10%  10-15%  15-20%  20-25%  >25% 

Percentage  cost  change  interval 


Amount  of  cost  change  in  each  percentage  interval  over  the  past  year 
Number  of  programs  in  each  percentage  cost  change  interval 
Source:  GAO  analysis  of  Department  of  Defense  data.  |  GAO-1 7-333SP 

Note:  Any  percentage  that  is  an  exact  multiple  of  5  is  put  in  the  higher  category. 


Of  the  46  programs  that  experienced  cost  increases  in  the  past  year, 
27  realized  a  total  of  $4.6  billion  in  increases  that  were  not  due  to 
quantity  increases.  Another  18  programs  saw  their  costs  increase  by 
$22.2  billion  while  also  increasing  their  planned  quantities.  One 
program — the  Joint  Air-to-Surface  Standoff  Missile  had  a  cost 
increase  of  $59.7  million,  but  also  a  quantity  decrease. 

Of  the  32  programs  that  experienced  cost  decreases  in  the  past  year, 
24  realized  a  total  of  $10  billion  in  decreases  that  were  not  due  to 
quantity  decreases.  Another  7  programs  saw  decreases  of  $6.8  billion 
due  to  quantity  decreases,  while  one  program — AIM-120  Advanced 
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Medium  Range  Air-to-Air  Missile — had  a  cost  decrease  of  $654.7 
million  despite  an  increase  in  quantity. 

2.  Development  and  procurement  costs  increased  by  $3.7  and  $5.6 
billion,  respectively.  Three  programs — the  Evolved  Expendable 
Launch  Vehicle,  the  Next  Generation  Operational  Control 
System,  and  the  Littoral  Combat  Ship — account  for  $2.2  billion  of 
the  aggregate  development  cost  increase.  The  three  largest 
procurement  cost  increases  were  in  shipbuilding  programs, 
which  all  increased  their  quantities.  Table  1  outlines  the  changes  in 
funding  for  the  2016  portfolio. 


Table  1 :  Changes  in  DOD’s  2016  Portfolio  of  78  Major  Defense  Acquisition 

Programs  over  the  Past  Year 

Fiscal  year  2017  dollars  (in  billions) 

Estimated 
portfolio 
cost  in  2015 

Estimated 
portfolio 
cost  in  2016 

Estimated 
cost  change 
since  2015 

Percentage 
change 
since  2015 

Total  estimated  research 
and  development  cost 

286.0 

289.7 

3.7 

1.3 

Total  estimated 
procurement  cost 

1,156.3 

1,162.0 

5.6 

0.5 

Total  estimated  acquisition 
cost3 

1,455.4 

1,464.7 

9.4 

0.6 

Source:  GAO  analysis  of  DOD  data.  |  GAO-1 7-333SP 

Note:  Some  numbers  may  not  sum  due  to  rounding. 

aln  addition  to  research  and  development  and  procurement  costs,  total  acquisition  cost  includes 
acquisition-related  operation  and  maintenance  and  system-specific  military  construction  costs. 


The  current  portfolio’s  development  and  procurement  cost  increases 
over  the  past  year  resulted  in  a  total  increase  of  $9.4  billion.  Of  the  78 
programs,  52  increased  their  development  costs  over  the  past  year. 
Increases  for  18  of  these  programs  were  under  $10  million  each,  and 
the  remaining  range  between  $1 0  million  and  $1 .1  billion. 

The  three  largest  development  cost  increases  included  two  space 
programs  and  one  shipbuilding  program.  The  largest  development 
cost  increase  was  on  the  Evolved  Expendable  Launch  Vehicle 
program.  Its  cost  increased  by  $1 .1  billion  to  conduct  new  engine 
development  activities.  The  Next  Generation  Operational  Control 
System’s  costs  increased  by  $575  million  to  address  technical  issues 
in  its  development.  Further,  Littoral  Combat  Ship  development  costs 
increased  by  $469  million  to  fund  engineering  support,  test  and 
evaluation,  and  training  development  needs  in  support  of  its  new 
frigate-related  design  enhancements. 
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Three  shipbuilding  programs  increased  the  quantity  of  ships  they  plan 
to  buy,  which  increased  their  procurement  costs  and,  subsequently, 
total  acquisition  costs  significantly.  Total  acquisition  costs  for  the  DDG 
51  Arleigh  Burke  Class  Guided  Missile  Destroyer,  the  Littoral  Combat 
Ship,  and  the  SSN  774  Virginia  Class  Submarine  increased  by  $6.3 
billion,  $5.5  billion  and  $3.4  billion,  respectively.  These  increases  are 
in  contrast  to  the  decreases  in  procurement  costs  and  total  acquisition 
costs  that  we  observed  in  our  2016  assessment. 

3.  The  amount  of  future  funding  needed  to  complete  the  portfolio’s 
planned  development  activities  and  procure  all  planned  units — 
$573.6  billion — decreased.  Of  this  future  funding,  $546  billion  is 
for  procurement  and  $27.6  billion  is  for  development.  The 
remaining  funds  for  development  total  less  than  half  the  amount 
that  remained  10  years  ago.13  The  current  portfolio’s  average 
delay  to  delivering  capability  increased  by  almost  2  months  over 
the  past  year.  The  aforementioned  cost  changes  indicate  that  more 
of  the  portfolio’s  total  acquisition  cost  is  spent  and  unavailable  for 
allotment.  The  future  funding  available  to  conduct  planned 
development  activities  and  acquire  expected  units  is  40  percent  of  the 
total  development  and  procurement  funding  in  the  current  portfolio.  By 
contrast,  in  2006,  the  amount  of  future  funding  available  was  over 
$966  billion,  or  59  percent,  of  the  portfolio’s  total  development  and 
procurement  cost.  Figure  4  shows  the  change  in  the  portfolio’s  future 
funding  and  funding  invested  as  a  share  of  the  portfolio’s  total 
acquisition  cost  since  2006. 


13Other  costs  include  military  construction  and  acquisition  operations  and  maintenance, 
which  account  for  $13.1  billion  of  the  2016  portfolio’s  total  acquisition  cost. 
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Figure  4:  DOD  Portfolio  Future  Development  and  Procurement  Funding  in  Comparison  to  Invested  Funding  by  Year,  2006- 
2016 
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Note:  DOD  did  not  issue  SARs  in  2009,  which  precludes  us  from  having  the  cost  baseline  information 
necessary  to  include  2009  in  this  analysis. 


In  addition,  the  amount  of  future  development  funding  in  the  current 
portfolio  is  less  than  half  of  what  it  was  10  years  ago.  The  $27.6  billion 
in  future  development  funding  is  4.8  percent  of  the  available  future 
funding,  compared  to  the  amount  available  in  2006,  which  was  over 
11.8  percent. 
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Some  programs’  costs  increased  due  to  schedule  delays.  When 
measuring  the  current  portfolio’s  schedule  performance  over  the  past 
year,  we  found  that  initial  operational  capability  delays  increased  by 
almost  2  months  and  now  stand  at  30.8  months  cumulative  beyond 
initial  estimates.14  Of  the  78  programs  in  the  201 6  portfolio,  1 1 
suffered  delays  that  contributed  to  the  overall  portfolio  delay.  Seven  of 
these  1 1  programs  reported  a  delay  of  6  months  or  more — delays 
significant  enough  to  require  DOD  to  report  them  as  a  breach  to  the 
acquisition  program  baseline.  The  largest  delay — 2.5  years — was  in 
the  Remote  Minehunting  System  program,  which  DOD  has  since 
canceled. 


Factors  that  explain  the 
changes 


4.  The  entire  portfolio  has  experienced  over  $484  billion  in  cost 
growth  since  programs  established  their  first  full  estimates.15 
However,  $476  billion  of  this  growth  occurred  5  or  more  years 
ago.  The  portfolio  realized  a  majority  of  its  cost  growth,  or  $259 
billion,  after  production  start.  Programs  started  before  2010 
make  up  a  majority  of  the  total  cost  growth  since  first  full 
estimates.  At  the  start  of  system  development  and  with  the  approval 
of  the  Milestone  Decision  Authority,  programs  establish  an  acquisition 
program  baseline  to  measure  their  performance  against.  This  baseline 
includes  a  cost  estimate  for  the  proposed  solution  that  identifies 
estimated  costs  over  the  full  life  of  the  program,  including 
development  and  production.  Our  analysis  of  the  2016  portfolio  found 
that  programs’  costs  grow  throughout  development,  but  that  a  majority 
of  this  cost  growth  occurs  after  programs  have  started  production. 
Figure  5  illustrates  the  portfolio’s  development  and  procurement  cost 
growth  as  incurred  between  key  program  milestones. 


14When  calculating  this  delay,  we  obtained  schedule  information  for  the  cycle  time  from 
program  start  to  initial  operational  capability  as  reported  in  the  previous  year  and 
contrasted  it  with  the  current  schedule. 

15The  first  full  estimate  is  generally  the  cost  estimate  established  at  the  start  of  system 
development,  for  more  information  see  Appendix  I.  For  more  information  on  the  portfolio's 
performance  since  first  full  estimates,  see  Appendix  III. 
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Figure  5:  Development  and  Procurement  Cost  Growth  between  Key  Milestones  for 
Programs  Included  in  DOD’s  2016  Portfolio 
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Source:  GAO  analysis  of  Department  of  Defense  data.  |  GAO-1 7-333SP 

Note:  To  accommodate  shipbuilding  programs  in  this  analysis,  we  correlated  detail  design  contract 
awards,  fabrication  starts,  and  lead  ship  deliveries  with  development  start,  critical  design  review,  and 
production  start,  respectively. 


In  the  first  phase  of  development,  between  the  start  of  a  program  and 
critical  design  review,  programs  realized  $77.7  billion  in  development 
and  procurement  cost  growth,  $25.7  billion  of  which  was  solely 
development  cost  growth.  In  the  next  phase,  or  between  the  critical 
design  review  and  the  initial  production  decision,  programs  realized 
an  additional  $90.7  billion  in  cost  growth — $21  billion  in  development 
funding  and  $69.7  billion  in  procurement  funding.  Finally,  after  their 
production  decisions,  programs  realized  another  $253.6  billion  in  cost 
growth,  which  is  a  60  percent  increase  over  their  initial  development 
and  procurement  estimates. 

A  majority  of  programs’  development  and  procurement  cost  growth 
occurs  after  they  have  started  production.  These  significant  post¬ 
production  cost  increases — particularly  within  development  funds — 
may  indicate  that  programs  start  production  without  having 
demonstrated  that  a  fully  integrated,  capable  production- 
representative  prototype  will  work  as  intended.  Alternatively,  post¬ 
production  increases  to  procurement  costs  within  programs  may  be 
explained,  in  part,  by  unplanned  quantity  increases. 

5.  The  portfolio  realized  a  buying  power  gain  of  $10.7  billion, 

although  more  programs  overall  lost  buying  power  over  the  past 
year.  To  understand  the  changes  in  the  portfolio’s  total  cost  over  the 
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past  year,  the  effect  of  changes  in  quantity  on  individual  programs 
must  be  analyzed  and  understood.  In  general,  buying  power  can  be 
defined  as  the  amount  of  goods  or  services  that  can  be  purchased 
given  a  specified  level  of  funding.  To  determine  changes  in  buying 
power,  the  effects  of  quantity  changes  must  be  isolated  from  other 
factors  that  affect  cost.  For  example,  a  program’s  cost  can  increase 
solely  because  of  additional  quantities.  While  that  does  represent  a 
cost  increase,  it  does  not  necessarily  indicate  acquisition  problems  or 
a  loss  of  buying  power.  Alternatively  a  program’s  cost  can  decrease 
due  to  a  reduction  in  quantity  and  may  still  experience  a  buying  power 
gain  or  loss.  Table  2  shows  our  calculation  of  how  programs’  cost  and 
quantity  changes  affected  their  buying  power. 


Table  2:  Buying  Power  Analysis  for  the  2016  Portfolio 

Fiscal  year  2017  dollars,  billions 

Number  of 
programs 

GAO  calculated 
cost  change  due 
to  quantity 
changes 

Actual 
procurement 
cost  change 

GAO  calculated 
cost  change  not 
attributable  to 
quantity  changes 

Programs  that  gained  buying  power 

33 

1.8 

-15.1 

-16.9 

Procurement  cost  decreased  with  no  quantity  change 

24 

0.0 

-13.6 

-13.6 

Quantity  increased  with  less  cost  increase  than 
anticipated 

6 

3.4 

1.5 

-2.0 

Quantity  decreased  with  more  cost  decrease  than 
anticipated 

3 

-1.7 

-3.1 

-1.4 

Programs  that  lost  buying  power 

40 

14.6 

20.7 

6.2 

Procurement  cost  increased  with  no  quantity  change 

25 

0.0 

2.6 

2.6 

Quantity  increased  with  more  cost  increase  than 
anticipated 

12 

15.8 

19.2 

3.4 

Quantity  decreased  with  less  cost  decrease  than 
anticipated 

3 

-1.2 

-1.1 

0.2 

Programs  with  no  change  in  buying  power 

5 

0.0 

0.0 

0.0 

Portfolio  totals 

78 

16.3 

5.6 

-10.7 

Source:  GAO  analysis  of  DOD  data.  |  GAO-1 7-333SP 


Note:  GAO  calculated  cost  change  “due”  to  quantity  changes  as  the  change  in  quantity  over  the  last 
year  multiplied  by  the  average  procurement  unit  cost  and  we  calculated  the  cost  change  “not  due”  to 
quantity  changes  as  the  current  acquisition  quantity  times  the  change  in  average  per  unit  costs.  If 
changes  in  quantity  affect  per  unit  cost,  those  changes  will  appear  in  the  cost  change  “not  due”  to 
quantity  changes  column. 


According  to  our  calculation  of  cost  versus  quantity  changes,  we 
would  have  expected  procurement  costs  in  the  portfolio  to  increase  by 
$16.3  billion.  Instead,  procurement  costs  for  the  portfolio  have 
increased  by  $5.6  billion  over  the  past  year,  indicating  that  programs 
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found  efficiencies  in  other  areas  to  offset  the  buying  power  losses. 

This  net  result  is  largely  due  to  Army  and  Air  Force  programs  gaining 
buying  power  and  is  offset  by  buying  power  losses  in  Navy  programs. 

Our  analysis  shows  that  33  programs  increased  in  buying  power  over 
the  past  year  and  reduced  procurement  costs  by  a  total  of  $15.1 
billion.  This  total  is  the  net  amount  of  cost  change  given  increases  and 
decreases  due  to  other  program  efficiencies.  Twenty-four  of  these  33 
programs  decreased  procurement  costs  with  no  change  in  their 
procurement  quantity,  indicating  that  they  found  efficiencies 
elsewhere.  For  example,  the  2  DOD-lead  programs — F-35  Lightning  II 
and  the  Joint  Light  Tactical  Vehicle — had  significant  decreases  in  total 
cost  not  due  to  quantity  changes,  which  also  drove  the  overall  buying 
power  gain.  Some  of  these  programs  reported  identifying  significant 
realized  and  expected  “should-cost”  savings  that  resulted  in  a 
reduction  of  procurement  cost  or  offset  the  cost  of  additional 
quantities. 

Six  programs  are  buying  additional  quantities  at  lower  prices.  In  other 
words,  their  planned  procurement  quantities  increased,  but  the 
corresponding  procurement  cost  did  not  increase  or  was  offset,  at 
least  in  part,  by  other  efficiencies.  For  example,  the  M109A7  Family  of 
Vehicles  program  plans  to  procure  12  additional  vehicles.  Our 
analysis  indicates  that  if  the  cost  increase  was  what  we  expected,  the 
program’s  procurement  costs  should  have  risen  by  $132  million. 
Instead,  the  program’s  procurement  cost  only  increased  by  $123 
million  and  the  program  realized  a  buying  power  gain. 

Conversely,  40  programs  lost  buying  power  in  the  past  year  with 
actual  procurement  cost  increases  of  $20.7  billion.  By  our 
calculations,  the  net  result  of  quantity  changes  on  these  programs 
should  have  resulted  in  a  $14.6  billion  cost  increase.  Procurement 
cost  increases  not  related  to  quantity  changes  generated  $2.6  billion 
in  additional  costs  and  a  net  buying  power  loss.  Contributing  to  this 
were  25  programs  that  lost  buying  power  as  their  procurement  costs 
increased  with  no  change  in  quantities,  an  indication  of  inefficiencies 
within  these  programs.  For  example,  the  Joint  Tactical  Radio  System 
Handheld,  Manpack,  and  Small  Form  Fit  Radios  program  lost  buying 
power  as  it  experienced  a  procurement  cost  increase  of  $310  million 
over  the  past  year  with  no  change  in  its  planned  procurement 
quantity.  Twelve  programs  increased  their  planned  procurement 
quantities  but  incurred  a  higher  than  expected  procurement  cost 
increase,  indicating  that  they  lost  efficiencies  elsewhere.  The 
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remaining  3  programs  reported  decreases  in  their  initial  quantities; 
however,  the  cost  reductions  on  these  programs  are  less  than 
expected  by  our  calculations. 

6.  As  measured  against  specific  cost  growth  metrics,  78  percent  of 
programs  meet  the  threshold  for  less  than  10  percent  cost 
growth  over  the  past  5  years.  However,  fewer  programs  meet  the 
metrics  for  realizing  less  than  2  percent  cost  growth  over  1  year 
or  for  less  than  15  percent  cost  growth  from  initial  estimates.  In 
December  2008,  we  discussed  and  agreed  with  DOD  and  the  Office 
of  Management  and  Budget  on  a  set  of  outcome  metrics  and  goals  to 
measure  program  cost  performance  over  time.  The  metrics  are 
intended  to  measure  cost  performance  on  a  percentage  basis  over 
three  defined  periods:  the  preceding  year,  the  preceding  5  years,  and 
since  first  full  estimates  were  established.  We  have  reported  on  these 
outcomes  since  201 1  and  figure  6  shows  how  the  performance  of  the 
current  portfolio  compares  to  our  prior  assessments. 


Figure  6:  Comparison  of  the  Cost  Performance  of  DOD’s  2012-2016  Portfolios 
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Note:  DOD  did  not  issue  Selected  Acquisition  Reports  in  2009,  which  precludes  us  from  having  the 
cost  baseline  information  necessary  to  assess  the  5-year  performance  of  the  2014  portfolio. 
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- First  full  estimate  comparison - 

(<15%  growth) 
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We  found  that  78  percent  of  programs  kept  their  cost  growth  under  10 
percent  over  the  past  5  years,  which  is  a  six  point  improvement  over 
the  2015  portfolio.  The  number  of  programs  in  the  portfolio  meeting 
this  metric  since  2013  has  been  increasing.  While  the  5-year  trend  is 
encouraging,  less  than  half  of  the  programs  in  the  portfolio  are  able  to 
keep  their  cost  growth  under  15  percent  from  their  initial  cost 
estimates.  Similarly,  the  number  of  programs  in  the  2016  portfolio  that 
meet  the  1-year  metric  by  limiting  their  total  acquisition  cost  growth  to 
less  than  2  percent  decreased  from  76  percent  to  72  percent. 


Post-reform  program  7.  Nineteen  programs  started  development  after  the  implementation 

observations  of  acquisition  reforms  in  2010.  These  programs  decreased  their 

costs  by  $3.4  billion  over  the  past  year — continuing  a  trend  that 
began  in  2015 — but  experienced  an  average  delay  to  delivering 
capability  of  over  3  months.  Since  2010,  DOD  has  taken  steps  to 
improve  the  outcomes  of  defense  acquisition  programs  including 
implementing  the  reforms  in  WSARA  and  DOD’s  “Better  Buying 
Power”  initiatives.16  As  a  group,  these  19  programs  realized  an  overall 
cost  decrease,  but  also  a  schedule  delay.  The  cost  decrease  is 
primarily  due  to  a  large  decrease  on  one  program — the  Joint  Light 
Tactical  Vehicle — and  the  net  result  of  cost  changes  on  the  remaining 
18  programs.  During  the  past  year,  9  programs  reported  cost 
decreases  totaling  $4.8  billion,  while  10  programs  reported  cost 
increases  totaling  approximately  $1.4  billion;  together,  this  resulted  in 
an  overall  cost  decrease  of  $3.4  billion.  Table  3  compares  statistics 
for  programs  in  the  2016  portfolio  that  began  before  2010  (59)  to 
statistics  for  the  programs  initiated  in  or  after  2010  (19),  once  reforms 
were  implemented. 


Table  3:  Observations  on  Programs  Initiated  before  2010  and  in  or  after  2010  That  Together  Comprise  DOD’s  2016  Portfolio 

(All  dollars  in  fiscal  year  201 7  dollars) 


Programs  initiated  before  2010  Programs  initiated  in  or  after  2010 


Number  of  programs 

59 

Number  of  programs 

19 

Total  acquisition  cost 

1.32  trillion 

Total  acquisition  cost 

147.1  billion 

1-year  cost  change 

12.8  billion  increase 

1-year  cost  change 

3.4  billion  decrease 

Cost  change  since  original  estimate 

486.3  billion  increase 

Cost  change  since  original  estimate 

3.5  billion  decrease 

Percentage  of  programs  with  cost 
increases  over  the  past  year 

61 

Percentage  of  programs  with  cost 
increases  over  the  past  year 

53 

16Many  of  these  reforms  and  initiatives  have  been  incorporated  into  DOD  Instruction 
5000.02. 
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(All  dollars  in  fiscal  year  201 7  dollars) 

Programs  initiated  before  2010 

Programs  initiated  in  or  after  2010 

Total  1-year  cost  increases 

25.4  billion 

Total  1-year  cost  increases 

1 .4  billion 

Percentage  of  programs  with  cost 
decreases  over  the  past  year 

39 

Percentage  of  programs  with  cost 
decreases  over  the  past  year 

47 

Amount  of  1  -year  cost  decreases 

12.6  billion 

Amount  of  1  -year  cost  decreases 

4.8  billion 

Schedule  accelerated  over  the  past 
year 

1 .3  months 

Schedule  delay  over  the  past  year 

3.1  months 

Source:  GAO  analysis  of  DOD  data.  |  GAO-1 7-333SP 


The  total  acquisition  cost  of  the  59  and  19  programs,  respectively,  is 
$1 .32  trillion  and  $147.1  billion.  Of  these  amounts,  $257.1  billion  and 
$32.6  billion  are  development  funding,  or  roughly  20  percent  for  each 
group  of  programs.  Each  group  also  had  more  programs  with  cost 
increases  than  not,  and  each  generally  had  a  similar  percentage  of 
programs  with  cost  increases  and  decreases. 

Of  the  19  programs  initiated  in  2010  or  later,  3  reported  a  delay  of  1 
year  or  more  to  their  initial  operational  capability.  The  Next  Generation 
Operational  Control  System,  Joint  Light  Tactical  Vehicle,  and  Small 
Diameter  Bomb  Increment  II  programs  reported  delays  of  2  years, 
over  1  year,  and  1  year  respectively.  We  found  that  a  similar 
proportion  of  programs  initiated  before  2010  also  reported  delays. 
Schedule  delays  may  cause  a  program’s  cost  to  increase.  However, 
delays  may  also  indicate  that  the  program  recognizes  it  needs  more 
time  before  exiting  development  to  be  ready  to  continue  into  later 
phases,  such  as  testing  and  production. 

8.  These  19  programs  represent  about  one-quarter  of  the  programs 
in  the  whole  portfolio,  but  account  for  only  10  percent  of  the 
portfolio’s  total  acquisition  cost.  Many  of  the  programs  that  started 
system  development  in  or  after  2010  may  not  have  been  in 
development  long  enough  to  realize  similar  cost  growth  as  the  rest  of 
the  portfolio.  Nonetheless,  all  but  2  of  these  programs — the  Combat 
Rescue  Helicopter  and  the  Intercontinental  Ballistic  Missile  Fuze 
Modernization — have  held  a  critical  design  review  or  started  system 
development  concurrent  with  a  production  decision  and  are  now  at  a 
point  in  the  acquisition  cycle  where  other  programs  have  experienced 
significant  cost  growth,  as  we  identified  above.  Nine  of  the  19 
programs  held  a  critical  design  review  in  the  past  3  years.  The 
estimated  cost  for  these  19  programs  is  less  than  half  the  cost  of  the 
F-35  Lightning  II,  as  shown  in  figure  7  below. 
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Figure  7:  Ten  Percent  of  DOD’s  Total  Acquisition  Cost  Is  for  the  19  Programs 
Initiated  in  or  after  2010  and  90  Percent  Is  for  the  Remaining  58  Programs  and  the  F- 
35  Lightning  II  (fiscal  year  2017  dollars  in  billions) 


23% 

of  the  total  acquisition 
cost  is  for  F-35  Lightning  II 


10% 

of  the  total  acquisition 
cost  is  for  19  programs 
initiated  in  or  after  2010 


67% 

of  the  total  acquisition  cost  is 

for  58  programs  initiated  before  2010 


Source:  GAO  analysis  of  Department  of  Defense  data.  |  GAO-1 7-333SP 


9.  Over  the  past  year,  total  acquisition  cost  changes  for  these  19 
programs  show  a  similar  profile  to  the  cost  changes  affecting  the 
remaining  59  programs.  Further,  an  inordinately  large  program  in 
each  group — Joint  Light  Tactical  Vehicle  and  F-35  Lightning  II — 
drives  a  majority  of  the  overall  cost  change.  We  analyzed  the  cost 
and  schedule  changes  for  the  19  programs  started  after  acquisition 
reforms  to  compare  their  performance  as  a  group  to  the  remaining  59 
programs  in  the  portfolio.  We  found  that,  in  many  ways,  they  are 
performing  similarly  in  terms  of  cost.  For  example,  a  majority  of 
programs  in  each  group  saw  small  percent  changes  in  their  total 
acquisition  cost  growth  resulting  in  similarly  shaped  bell  curves  in 
figure  8.  Figure  8  also  illustrates  the  distribution  of  total  acquisition 
cost  change  amount  in  each  percent  change  interval  over  the  past 
year  for  both  sets  of  programs. 
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Figure  8:  Distribution  of  the  1-year  Change  in  Total  Acquisition  Cost  for  the  Programs  Initiated  before  2010  and  the  Programs 
Initiated  in  or  after  2010 


Fiscal  year  2017  dollars  (in  billions) 


Number  of  programs 


Amount  of  cost  change  in  each  percentage  interval  over  the  past  year  for  programs  initiated  before  2010 
Amount  of  cost  change  in  each  percentage  interval  over  the  past  year  for  programs  initiated  in  or  after  2010 


Number  of  programs  in  each  percentage  change  cost  interval  for  programs  initiated  before  2010 


Number  of  programs  in  each  percentage  change  cost  interval  for  programs  initiated  in  or  after  2010 
Source:  GAO  analysis  of  Department  of  Defense  data.  |  GAO-1 7-333SP 

Note:  Any  percentage  that  is  an  exact  multiple  of  5  is  put  in  the  higher  category. 


The  distribution  curve  of  each  set  of  programs  is  similar  with  a 
majority  of  programs  between  a  5  percent  decrease  and  5  percent 
increase  over  the  past  year  and  only  a  few  programs  with  large 
percent  changes.  We  also  identified  the  Joint  Light  Tactical  Vehicle 
and  F-35  Lightning  II  as  outlier  programs.  These  2  programs  had 
significant  cost  decreases  over  the  past  year  that  disproportionately 
affected  the  net  cost  change  for  their  respective  groups. 
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Other  observations 


10.  Forty-nine  percent  of  programs  intend  to  or  have  declared  initial 
operational  capability  on  the  basis  of  limited  or,  in  a  few  cases, 
no  operational  testing.  Specifically,  9  percent  declared  initial 
operational  capability  before  beginning  testing,  and  40  percent 
did  or  plan  to  declare  initial  operating  capability  before  testing 
completes.  Programs  declaring  initial  operational  capability  prior 
to  completing  initial  operational  test  and  evaluation  risk  fielding 
systems  that  are  later  found  to  be  ineffective  or  unsuitable  for 
missions.  According  to  DOD  policy,  the  appropriate  authority  will 
declare  initial  operational  capability  when  the  unit  or  organization  has 
been  equipped  and  trained  and  is  determined  to  be  capable  of 
conducting  mission  operations.  The  specifics  for  a  particular  system’s 
initial  operational  capability  are  defined  in  its  Capability  Development 
Document  and  Capability  Production  Document,  which  DOD  requires 
before  a  program  enters  system  development  and  production, 
respectively.  For  instance,  when  drafting  a  Capability  Development 
Document,  programs  describe  the  types  and  initial  quantities  of  assets 
required  to  attain  initial  operational  capability.  They  also  identify  the 
operational  units  such  as  the  service  or  other  government  agencies 
that  will  employ  the  capability.  Initial  operational  capability  definitions 
in  these  documents,  however,  may  or  may  not  include  the  Director, 
Operational  Test  and  Evaluation’s  assessment  of  the  system’s 
effectiveness  and  suitability. 

Typically,  initial  operational  capability  and  operational  test  and 
evaluation  both  occur  during  a  program’s  production  and  deployment 
phase.  Initial  operational  test  and  evaluation  is  intended  to  evaluate  a 
system’s  effectiveness  and  suitability  under  realistic  combat 
conditions  before  a  full-rate  production  decision.  Flowever,  DOD’s 
Instruction  5000.02  may  be  tailored  to  meet  program  objectives,  and 
DOD’s  Test  and  Evaluation  Management  Guide  notes  that  initial 
operational  capability  is  usually  determined  by  the  service. 
Consequently,  programs  can  declare  initial  operational  capability  on 
the  basis  of  full,  partial,  or  no  initial  operational  test  and  evaluation. 

Our  analysis  found  that  nearly  half  the  programs  in  the  current 
portfolio  declared,  or  plan  to  declare,  initial  operational  capability 
before  or  during  testing.  Figure  9  illustrates  our  analysis  of  the 
relationship  between  initial  operational  capability  and  initial 
operational  test  and  evaluation  in  current  DOD  programs. 
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Figure  9:  Programs’  Timing  of  Declaring  Initial  Operational  Capability  and  Completing  Initial  Operational  Test  and  Evaluation 


1  Operational  testing  start 

|  Operational  testing  complete 

Not  applicable 

16 

Capability 
declared 
before 
testing  -  7 

Capability  declared  during  testing 

31 

Capability  declared  after  testing 

24 

Cooperative  Engagement  Capability  (CEC) 

Combat  Rescue  Helicopter  (CRH) 

Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier  (CVN  78) 
Global  Broadcast  Service  (GBS) 

Ground/Air  Task  Oriented  Radar  (G/ATOR) 

Littoral  Combat  Ship  (LCS) 

SSN  774  Virginia  Class  Submarine  (SSN  774) 


AGM-88E  Advanced  Anti-Radiation  Guided  Missile  (AGM-88E  AARGM) 

AH-64E  Apache  New  Build  (AH-64E  New  Build) 

AH-64E  Apache  Remanufacture  (AH-64E  Remanufacture) 

Armored  Multi-Purpose  Vehicle  (AMPV) 

B-2  Extremely  High  Frequency  SATCOM  and  Computer  Increment  1  (B-2  EHF  Incl) 

DDG  1000  Zumwalt  Class  Destroyer  (DDG  1000) 

DDG  51  Arleigh  Burke  Class  Guided  Missile  Destroyer  (DDG  51) 

E-2D  Advanced  Hawkeye  Aircraft  (E-2D  AHE) 

EA-18G  Growler  Aircraft  (EA-18G) 

F-22  Increment  3.2B  Modernization  (F-22  Inc  3.2B  Mod) 

F-35  Lightning  II  Program  (F-35) 

HC/MC-130  Recapitalization  Aircraft  (HC/MC-130  Recap) 

Integrated  Defensive  Electronic  Countermeasures  (1DECM)  Block  4 
Joint  Air-to-Ground  Missile  (JAGM) 

Joint  Precision  Approach  and  Landing  System  Increment  1 A  (JPALS  Inc  1  A) 

Joint  Tactical  Radio  System  Handheld,  Manpack,  and  Small  Form  Fit  Radios  (JTRS  HMS) 
KC-46  Tanker  Modernization  Program  (KC-46A) 

LHA  6  America  Class  Amphibious  Assault  Ship  (LHA  6) 

LPD  17  San  Antonio  Class  Amphibious  Transport  Dock(LPD  17) 

M109A7  Family  of  Vehicles  (M109A7  FOV) 

MH-60R  Multi-Mission  Helicopter  (MH-60R) 

MQ-4C  Triton  Unmanned  Aircraft  System  (MQ-4C  Triton) 

MQ-9  Reaper  Unmanned  Aircraft  System  (MQ-9  Reaper) 

Multifunctional  Information  Distribution  System  (MIDS) 

Navy  Multiband  Terminal  (NMT) 

Next  Generation  Operational  Control  System  (GPS  OCX) 

Patriot  Advanced  Capability-3  Missile  Segment  Enhancement  (PAC-3  MSE) 

Remote  Minehunting  System  (RMS) 

Ship  to  Shore  Connector  Amphibious  Craft  (SSC) 

Small  Diameter  Bomb  Increment  II  (SDB  II) 

Space  Fence  Ground-Based  Radar  System  Increment  1 


Source:  GAO  analysis  of  Department  of  Defense  data.  |  GAO-1 7-333SP 

Note:  The  quantity  allotments  within  the  figure  reflect  programs  that  have  declared  initial  operational 
capability  as  well  as  some  that  have  not  yet  declared  it,  but  have  identified  when  they  expect  to  make 
the  declaration  in  relation  to  their  initial  operational  test  and  evaluation  plans. 


Programs  declaring  initial  operational  capability  prior  to  completing  full 
operational  testing  risk  finding  deficiencies  in  testing  that  may  need  to 
be  corrected,  which  could  add  to  a  program’s  cost  and  schedule  post¬ 
production.  For  example,  the  Navy  declared  initial  operational 
capability  for  the  Littoral  Combat  Ship  in  2014  while  facets  of  the 
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ship’s  capabilities — including  most  mission  equipment — were  still 
undergoing  developmental  testing.  Essentially,  declaring  initial 
operational  capability  signals  that  the  ship  is  ready  to  conduct 
missions.  However,  in  the  case  of  the  Littoral  Combat  Ship,  we 
testified  in  December  2016  that  the  Navy  had  fallen  short  of 
demonstrating  that  either  of  the  program’s  two  seaframe  designs,  or 
its  three  mission  packages,  met  the  minimum  level  of  capability 
defined  at  the  beginning  of  the  program.17  Further,  since  declaring 
initial  operational  capability,  the  program  has  identified  significant 
performance  limitations  affecting  both  seaframes  and  mission 
packages,  consequently  leading  it  to  postpone  initial  operational  test 
and  evaluation.  These  deficiencies  have  contributed,  in  part,  to  the 
Navy  truncating  the  program  at  fewer  quantities  than  originally 
planned  and  moving  to  a  new  design  frigate. 

Although  many  programs  are  not  utilizing  their  operational  testing 
results  to  inform  their  initial  capability  declarations,  some  are.  Thirty- 
one  percent  of  programs  either  achieved  or  plan  to  achieve  initial 
operational  capability  after  they  have  completed  operational  testing. 
Sixteen  programs  either  do  not  track  an  initial  operational  capability 
date  or  have  completed  or  planned  operational  test  dates. 

11.  The  leading  companies  DOD  relies  on  to  develop  and  produce 
military  capabilities  have  consistently  shown  strong  market 
performance,  which  indicates  that  investors  expect  the 
performance  of  these  companies  to  be  strong  for  some  time  to 
come.  Stock  prices  for  these  companies  have  increased 
significantly  over  time — outperforming  other  market  sectors  over 
the  past  1  year  and  5  years.  These  sustained  increases  have 
occurred  irrespective  of  cost  and  schedule  outcomes  in  the  defense 
acquisition  portfolio,  which  these  companies  help  support.  DOD  relies 
on  the  defense  industrial  base  to  provide  capabilities  that  equip  forces 
to  meet  mission  requirements.  The  defense  industrial  base  consists  of 
privately  owned  companies  of  various  sizes  to  which  DOD  awards 
contracts  to  develop  and  produce  its  major  weapon  systems.  The 
defense  industrial  base  works  in  tandem  with  DOD  to  provide  key 
capabilities  to  our  warfighters. 

We  observed  that  regardless  of  whether  DOD  programs  experienced 
cost  increases,  decreases,  or  stability,  nearly  every  leading  defense 
contractors’  stock  price  generally  increased.  This  provides  an 


17GAO,  Littoral  Combat  Ship  and  Frigate:  Slowing  Planned  Frigate  Acquisition  Would 
Enable  Better-Informed  Decisions,  GAO-1 7-279T  (Washington,  D.C.:  Dec.  8,  2016). 


Page  27 


GAO-1 7-333SP  Assessments  of  Selected  Weapon  Programs 


indication  that  investors  expect  the  performance  of  these  companies 
to  be  strong  for  some  time  to  come.  We  selected  six  publicly-traded 
defense  contractors  developing  and  delivering  the  largest  DOD 
acquisition  programs  and  analyzed  their  stock  prices  and  their  DOD 
programs’  cost  changes  over  the  past  year  and  5  years.18  These 
contractors  account  for  73  percent  of  the  contracts  for  weapon 
systems  in  the  current  portfolio. 

In  comparison,  these  defense  companies’  stock  prices  have  grown  at 
consistently  higher  rates  over  the  past  year  and  5  years  than  the  S&P 
500  and  the  Industrials  sector  of  the  S&P  500. 19  DOD  also  reported 
last  year  that  its  major  defense  companies  have  remained  profitable, 
even  in  a  DOD  business  environment  that  has  increasingly  attempted 
to  more  closely  link  contractor  profits  to  performance  in  defense 
acquisition  programs.20 

Cost  growth  in  the  portfolio  is  undesirable  from  DOD’s  perspective  as, 
in  a  constrained  funding  environment,  DOD  must  make  tradeoffs  to 
fund  cost  growth  on  existing  programs,  rather  than  starting  new 
programs  or  upgrading  existing  programs.  For  defense  contractors, 
the  effects  of  cost  growth  are  partially  determined  by  the  type  of 
contract  awarded.  A  range  of  contract  types  may  be  used  that  vary 
the  cost  risk  assumed  by  contractors  and  the  government,  depending 
on  numerous  factors.  In  addition,  incentives  can  be  used  to  incentivize 
the  contractor  to,  among  other  things,  control  costs.  DOD  stated  in  its 
2016  annual  report  on  the  performance  of  the  Defense  Acquisition 
System  that,  “the  real  issue  is  how  effective  the  incentives  are  for 
each  contract  type  based  on  the  situation  at  hand.” 

For  example,  a  DOD  program  may  award  a  firm-fixed-price  contract  if 
the  expected  cost  of  the  effort  is  well  understood.  A  firm-fixed-price 
contract  provides  for  a  price  that  is  not  subject  to  change  on  the  basis 
of  the  contractor’s  cost  experience  in  performing  the  contract.  This 
type  of  contract  places  most  of  the  risk  for  cost  growth  on  the 
contractor  and  provides  incentive  for  the  contractor  to  control  costs 
and  perform  effectively.  Some  fixed-price  type  contracts  provide  for 
cost  incentives.  For  example,  we  found  in  April  2015  that  the  KC-46A 


18We  did  not  include  all  defense  contractors  nor  did  we  include  any  smaller  defense 
contractors  in  our  assessment. 

1  indices  used  are  based  on  Standard  &  Poor’s  Global  Industry  Classification  System. 

20Under  Secretary  of  Defense,  Acquisition,  Technology,  and  Logistics,  Performance  of  the 
Defense  Acquisition  System:  2016  Annual  Report  (Washington  D.C.:  Oct.  24,  2016). 
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Tanker  Modernization  (KC-46A)  program  is  being  developed  under  a 
fixed-price  incentive  (firm  target)  contract,  which  the  Air  Force 
awarded  to  Boeing  in  201 1  because  it  considered  integrating  largely 
mature  military  technologies  onto  an  aircraft  designed  for  commercial 
use  to  be  a  relatively  low  risk  effort.21  This  contract  type  provides  for 
adjusting  profit  and  establishing  the  final  contract  price  by  application 
of  a  formula  based  on  the  relationship  of  total  final  negotiated  cost  to 
total  target  cost.  The  final  price  is  subject  to  a  price  ceiling,  negotiated 
at  the  outset,  which  is  the  maximum  that  may  be  paid  to  the 
contractor,  except  for  any  adjustment  under  other  contract  clauses.  By 
December  2015,  both  the  contractor  and  government  estimated  that 
costs  would  be  over  the  contract  ceiling  price.  As  such,  Boeing  will 
generally  assume  all  responsibility  for  the  additional  costs. 

Alternatively,  a  program  might  award  a  cost-reimbursement  contract  if 
costs  are  less  known  and  there  is  more  risk  for  cost  growth.  The  cost- 
plus  incentive  fee  type  contract  is  the  most  common  cost- 
reimbursement  contract  type  awarded  in  the  current  portfolio.  Also, 
programs  may  use  a  combination  of  contract  types  to  take  advantage 
of  the  benefits  each  affords.  Theoretically,  no  matter  the  contract  type, 
if  the  contractor  performs  as  expected,  it  should  generally  earn  profits 
which  could  affect  the  contractors’  stock  prices.  Flowever,  there  are 
also  cases  in  which  the  contractor  may  underperform,  and 
subsequently  earn  fewer  profits  than  expected. 

The  contractors  in  our  analysis  have  business  outside  of  their  DOD 
contracts  that  also  factor  into  their  overall  market  performance.  As 
such,  we  are  not  correlating  the  contractors’  performance  on  their 
DOD  contracts  with  their  stock  prices. 


Observations  about 
DOD’s 

Implementation  of 
Key  Acquisition 
Reform  Initiatives 


In  2016,  we  found  that  more  DOD  programs  were  implementing  WSARA 
acquisition  reforms  and  “Better  Buying  Power”  initiatives,  which  led  to 
better  acquisition  outcomes  on  some  programs. 

In  our  work  this  year,  we  focused  on  aspects  of  WSARA  and  DOD’s 
“Better  Buying  Power”  initiatives  that  address  program  and  portfolio 
affordability,  cost  growth  controls,  and  the  use  of  competition  throughout 
the  acquisition  life-cycle. 


21  GAO,  KC-46  Tanker  Aircraft:  Key  Aerial  Refueling  Capabilities  Should  Be  Demonstrated 
Prior  to  the  Production  Decision,  GAO-15-308  (Washington,  D.C.:  Apr.  9,  2015). 
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We  found  that  the  number  of  programs  implementing  reforms  is  similar  or 
slightly  less  than  what  we  have  observed  previously.  For  example,  over 
the  past  year,  38  programs  have  conducted  an  affordability  analysis  and 
set  affordability  constraints  whereas  16  have  not.  Also,  a  similar  number 
of  programs  have  conducted  a  “should-cost”  analysis.  These  programs 
reported  a  larger  total  amount  of  realized  and  future  expected  “should- 
cost”  savings  than  we  observed  previously,  totaling  $1 1 1 .5  billion.  This 
year,  more  programs  than  last  year  reported  having  no  plans  for 
competition  at  any  point  in  their  life  cycle.  Lastly,  we  assessed  programs’ 
plans  to  complete  an  evaluation  of  potential  cyber  vulnerabilities  by  2019. 

Our  analysis  allows  us  to  make  the  following  four  observations 
concerning  key  acquisition  reform  initiatives: 


Acquisition  Reform  Observations 

1 .  Of  the  54  current  and  future  programs  we  assessed,  38  have  established  an 
affordability  constraint  while  16  have  not.  This  is  similar  to  our  prior  assessment. 
Thus  far,  all  but  one  current  program  that  conducted  an  analysis  and  set  a  constraint 
reported  they  are  on  track  to  remain  within  their  constraints. 

2.  Of  the  45  current  programs,  42  conducted  a  “should-cost”  analysis,  and  41  of  those 
reported  anticipated  savings  of  $1 1 1 .5  billion;  $23.6  billion  of  these  savings  have 
been  realized  to  date.  Programs  we  assessed  were  unable  to  account  for  the 
recipient  of  roughly  half,  or  $1 1  billion,  of  the  realized  savings  that  they  reported. 

3.  Current  programs  are  conducting  competitive  prototyping  at  a  higher  rate  than  future 
programs  plan  to  do.  Further,  of  the  54  current  and  future  programs  we  assessed, 

41  plan  to  promote  some  means  of  competition  during  the  acquisition  cycle,  while  13 
have  no  plans  for  competition  before  or  after  development  start. 

4.  Of  the  45  current  programs  we  assessed,  31  reported  that  they  plan  to  complete  an 
evaluation  of  potential  cyber  vulnerabilities  by  2019. 

Source:  GAO  analysis  of  DOD  data.  |  GAO-1 7-333SP 


1 .  Of  the  54  current  and  future  programs  we  assessed,  38  have 
established  an  affordability  constraint  while  16  have  not.  This  is 
similar  to  our  prior  assessment.  Thus  far,  all  but  one  current 
program  that  conducted  an  analysis  and  set  a  constraint 
reported  they  are  on  track  to  remain  within  their  constraints.  One 
of  the  reform  initiatives  in  DOD’s  “Better  Buying  Power”  memoranda  is 
conducting  an  affordability  analysis  that  results  in  setting  program  cost 
constraints.  This  analysis  differs  from  program  cost  estimates  in  that 
the  constraint  serves  as  a  program  requirement  to  ensure  that  it 
remains  cost-effective. 

In  accordance  with  DOD  Instruction  5000.02,  affordability  constraints 
are  intended  to  force  prioritization  of  requirements,  enable  cost  trades, 
and  ensure  that  unaffordable  programs  do  not  enter  the  acquisition 
process.  When  approved  affordability  constraints  cannot  be  met,  a 
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program’s  technical  requirements,  schedule,  and  required  quantities 
must  be  revisited.  Failure  to  remain  within  these  constraints  may 
result  in  program  termination. 

Of  the  current  and  future  programs  we  assessed,  38  of  54  have 
established  an  affordability  constraint  while  16  have  not.  This  is  a 
similar  number  of  programs  that  had  implemented  affordability 
constraints  in  our  last  assessment. 

Of  the  16  programs  that  have  not  established  an  affordability 
constraint,  5  are  future  programs  that  have  not  completed 
development  start  at  which  point  the  constraints  are  put  in  place;  1 1 
are  current  programs,  6  of  which  began  system  development  before 
this  requirement  was  established  and  one  program  that  reported  its 
development  start  as  to  be  determined  (TBD).  One  program,  the  Next 
Generation  Operational  Control  System,  responded  that  it  does  not 
currently  expect  to  meet  its  affordability  constraints. 

While  the  effectiveness  of  these  constraints  has  yet  to  be  widely 
tested,  we  observed  that  the  current  programs  we  assessed  with 
established  affordability  constraints  had  a  lower  average  amount  of 
cost  growth  from  their  initial  estimates  compared  to  programs  without 
a  constraint.  Specifically,  of  the  45  current  programs,  1 1  programs 
without  a  constraint  saw  their  costs  increase  from  initial  estimates  by 
$1 .8  billion  whereas  the  34  programs  with  constraints  saw  their  costs 
increase  by  $1 .2  billion. 

2.  Of  the  45  current  programs,  42  conducted  a  “should-cost” 
analysis,  and  41  of  those  reported  anticipated  savings  of  over 
$111.5  billion;  $23.6  billion  of  these  savings  have  been  realized 
to  date.  Programs  we  assessed  were  unable  to  account  for  the 
recipient  of  roughly  half,  or  $11  billion,  of  the  realized  savings 
that  they  reported.  Stemming  from  DOD’s  “Better  Buying  Power” 
memoranda,  and,  in  accordance  with  DOD  Instruction  5000.02,  each 
program  must  conduct  a  “should-cost”  analysis  resulting  in  an 
estimate  to  be  used  as  a  management  tool  to  control  and  reduce  cost. 
“Should-cost”  analysis  can  be  used  to  justify  each  cost  under  the 
program’s  control  with  the  aim  of  reducing  negotiated  prices  for 
contracts  and  obtaining  other  efficiencies  in  program  execution  to 
bring  costs  below  those  budgeted  for  the  program.  Any  savings 
achieved  can  then  be  reallocated  within  the  program  or  for  other 
priorities. 

Programs  can  do  a  variety  of  things  that  could  lead  to  “should-cost” 
savings  including  multi-year  pricing,  acting  as  a  system  integrator, 
achieving  more  economical  productions  rates,  reducing  overhead, 
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and  improving  supply  chain  management,  among  others.  The  41 
programs  that  reported  anticipated  “should-cost”  savings  most 
frequently  cited  the  following  activities  as  responsible  for  some  of 
these  savings,  including: 

•  efficiencies  realized  through  contract  negotiations  (14  programs), 

•  design  trades  to  balance  affordability  and  capability  (1 1 
programs),  and 

•  developmental  or  operational  testing  efficiencies  (10  programs). 

Of  the  current  45  programs,  28  reported  $23.6  billion  in  realized 
“should-cost”  savings  to  date.  Of  this  amount,  $4.4  billion  in  savings 
accrued  from  reductions  in  development  costs,  $18.9  billion  from 
reductions  in  procurement  costs,  and  $0.2  billion  from  reductions  in 
military  construction  and  acquisition-related  operations  and 
maintenance  costs.  Programs  also  provided  insights  as  to  how  their 
realized  savings  have  been  allocated,  as  shown  in  figure  10. 


Figure  10:  Recipients  of  the  $23.6  Billion  in  Total  Realized  “Should-cost”  Savings 
(Fiscal  year  2017  dollars,  in  billions) 


Other  priorities 
Kept  in  program 

Went  to  outside  service  priorities 
Went  to  service  priorities 
Recipient  unknown 


Source:  GAO  analysis  of  Department  of  Defense  data.  |  GAO-1 7-333SP 
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Of  the  $23.6  billion  in  realized  “should-cost”  savings,  programs  could 
not  identify  where  almost  half,  or  $1 1  billion,  of  the  total  savings  was 
transferred  within  DOD.  DOD  programs  may  not  have  strong 
incentives  to  continue  to  identify,  realize,  and  report  “should-cost” 
savings  if  those  programs  do  not  know  what  the  funds  are  being  used 
for,  or  if  they  perceive  them  as  resulting  in  the  funding  of  other  DOD 
priorities.  Of  the  $1 2.6  billion  in  realized  savings  for  which  programs 
could  identify  the  recipients,  $2.1  billion  was  kept  within  the  programs 
to  fund  other  priorities.  Programs  reported  that  $178  million  of  savings 
realized  were  used  to  offset  budget  cuts  required  by  sequestration. 
Another  $6.1  billion  of  the  realized  “should-cost”  savings  went  to 
priorities  within  the  military  service  to  which  the  program  belongs,  and 
$3.3  billion  went  to  priorities  outside  of  the  service. 

Current  programs  that  we  surveyed  expect  to  realize  another  $87.9 
billion  in  future  “should-cost”  savings.  Of  this  amount,  programs 
expect  $1 .6  billion  and  $1 5.4  billion  in  savings  to  accrue  within 
development  and  procurement  funds,  respectively.  A  majority  of  the 
future  expected  savings  was  reported  as  acquisition-related 
operations  and  maintenance  savings — totaling  over  $70  billion.  Over 
$62  billion  of  the  $70  billion  was  reported  by  the  F-35  Lightning  II 
program.  In  addition  to  the  current  programs,  3  of  the  9  future 
programs  reported  having  conducted  a  “should-cost”  analysis  and 
identified  $7.5  billion  in  realized  and  future  savings. 

While  most  defense  programs  we  assessed  do  not  change  their  total 
cost  estimates  as  a  result  of  their  “should-cost”  analyses,  6  programs 
reported  that  their  total  cost  estimate  changed  as  a  result  of  the 
savings  identified,  and  2  more  programs  are  in  the  process  of  revising 
their  cost  estimate  to  account  for  their  “should-cost”  results. 

3.  Current  programs  are  conducting  competitive  prototyping  prior 
to  system  development  at  a  higher  rate  than  future  programs 
plan  to  do.  Of  the  54  current  and  future  programs  we  assessed, 

41  intend  to  promote  competition  at  some  point  during  the 
acquisition  cycle,  while  13  programs  have  no  plans  for 
competition  before  or  after  development  start.  Competition  is  a 
critical  tool  for  achieving  the  best  return  on  the  government’s 
investment.  Major  defense  acquisition  programs  are  encouraged  to 
include  in  their  acquisition  strategies  the  use  of  prototypes  from  two  or 
more  contractors  before  a  program  starts  system  development  and 
have  acquisition  strategies  that  ensure  the  option  of  continued 
competition  throughout  the  acquisition  life  cycle.  According  to  DOD, 
the  fostering  of  competitive  environments  is  a  central  tenet  in 
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acquisition  reform  and  the  single  best  way  to  motivate  contractors  to 
provide  the  best  value. 

Our  prior  work  has  shown  that  competitive  prototyping  can  help 
programs  reduce  technical  risk,  refine  requirements,  and  validate 
designs  and  cost  estimates  prior  to  making  major  commitments  of 
resources.22  The  National  Defense  Authorization  Act  for  Fiscal  Year 
2016  revised  the  requirement  for  programs  to  conduct  competitive 
prototyping  and  instead  emphasizes  prototyping  as  an  important  risk 
mitigation  approach.23  We  have  found  in  the  past  that  when  programs 
require  their  contractors  to  build  prototypes  for  early  demonstrations 
of  capability,  the  product  developers  complete  a  significant  amount  of 
systems  engineering  which  leads  to  valuable  knowledge  that  better 
ensures  that  the  requirements  can  be  met  within  available 
resources.24 

Of  the  45  current  programs,  just  under  half  conducted  competitive 
prototyping.  Yet,  of  the  9  future  programs,  only  one  identified  plans  to 
do  so  at  the  system  or  sub-system  level  before  the  start  of  system 
development.  Programs  not  conducting  competitive  prototyping  may 
be  missing  an  opportunity  to  lower  costs  and  reduce  technical  risk. 
Figure  1 1  shows  how  many  current  and  future  programs  plan  to 
conduct  competitive  prototyping. 


22GAO,  National  Defense:  Department  of  Defense’s  Waiver  of  Competitive  Prototyping 
Requirement  for  the  Navy’s  Fleet  Replenishment  Oiler  Program ,  GAO-1 5-57R 
(Washington,  D.C.:  Oct.  8,  2014). 

23Pub.  L.  No.  114-92,  §  822. 

24  GAO,  Weapon  System  Requirements:  Detailed  Systems  Engineering  Prior  to  Product 
Development  Positions  Programs  for  Success,  GAO-17-77  (Washington,  D.C.;  Nov.  17, 
201 6);  and,  Best  Practices:  Better  Matching  of  Needs  and  Resources  Will  Lead  to  Better 
Weapon  System  Outcomes ,  GAO-01 -288  (Washington,  D.C.:  Mar.  8,  2001). 
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Figure  11:  Competitive  Prototyping  Plans  among  54  Future  and  Current  Programs 

Number  of  programs 
25 


Source:  GAO  analysis  of  Department  of  Defense  data.  |  GAO-1 7-333SP 


Beyond  competitive  prototyping,  many  current  and  future  programs 
identified  plans  to  undertake  additional  measures  to  promote 
competition.  However,  compared  to  our  2016  assessment,  this  year 
more  programs  reported  having  no  plans  for  competition  before  or 
after  development  start.  Thirteen  programs  we  assessed — 6  future 
and  7  current  programs — are  not  planning  to  take,  or  took  no  actions, 
to  promote  competition  either  before  or  after  their  development  start 
dates.  Table  4  outlines  the  extent  to  which  current  and  future 
programs  are  planning  to  promote  competition  and  in  which  phases  of 
acquisition. 
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Table  4:  Actions  Reported  by  54  Current  and  Future  Programs  to  Promote 
Competition 

For  the  9 
future 
programs 

For  the  45 
current 
programs 

Total 

Number  of  programs  planning  to  promote 
competition 

3 

38 

41 

Throughout  acquisition  life  cycle 

1 

15 

16 

Only  prior  to  the  start  of  system 
development 

0 

7 

7 

Only  after  the  start  of  system  development 

2 

16 

18 

Number  of  programs  taking  no  actions  to 
promote  competition 

6 

7 

13 

Source:  GAO  analysis  of  DOD  data.  |  GAO-1 7-333SP 


4.  Of  the  45  current  programs  we  assessed,  31  reported  that  they 
plan  to  complete  an  evaluation  of  potential  cyber  vulnerabilities 
by  2019.  All  defense  acquisition  programs  containing  Information 
Technology  are  required  to  have  a  cybersecurity  strategy  that  is 
consistent  with  DOD  policy  and  standards.  In  addition  to  this  strategy, 
the  National  Defense  Authorization  Act  for  Fiscal  Year  2016  requires 
the  Secretary  of  Defense  to  complete  an  evaluation  of  the  cyber 
vulnerabilities  of  each  major  weapon  system  of  the  Department  of 
Defense  by  not  later  than  December  31 , 201 9. 25  In  201 6,  DOD 
reported  that  it  is  making  institutional  and  policy  changes  to  address 
cybersecurity,  among  other  things.  Specifically,  according  to  DOD,  it 
is  ensuring  that  cybersecurity  regulations  are  applied  to  contracts  to 
better  secure  controlled  information  in  the  defense  industry. 

Six  programs  identified  completion  dates  after  December  31, 2019. 
These  six  consist  of  several  programs  early  in  development,  including 
the  Three-Dimensional  Expeditionary  Long-Range  Radar,  F-15  Eagle 
Passive/Active  Warning  Survivability  System,  and  Next  Generation 
Jammer  Increments  1  and  2.  Two  additional  programs  identified 
completion  dates  that  correspond  with  when  they  expect  to  complete 
operational  testing — MQ-4C  Triton  Unmanned  Aircraft  System  in  2021 
and  VH-92A  Presidential  Helicopter  in  2020.  Another  program,  the 
LHA  6  America  Class  Amphibious  Assault  Ship,  currently  in 


25Pub.  L.  No.  1 1 4-92,  §  1 647.  The  requirement  may  be  fulfilled  after  201 9,  or  waived,  if 
the  Secretary  certifies  to  the  congressional  defense  committees  that  all  known  cyber 
vulnerabilities  have  minimal  consequences  for  the  capability  of  the  weapon  system  to 
meet  operational  requirements  or  otherwise  satisfy  mission  requirements. 
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production  and  deployment,  reported  a  date  corresponding  to  the  third 
ship  of  its  class.  Eight  other  programs  did  not  provide  dates  identifying 
when,  or  if,  they  would  complete  evaluations  of  cyber  vulnerabilities. 
One  of  these  programs — the  Air  and  Missile  Defense  Radar — 
reported  that  it  is  not  scheduled  to  complete  an  evaluation. 


Observations  from 
Our  Assessment  of 
Knowledge  Attained 
by  Programs  at  Key 
Junctures 


Our  analysis  found  that  although  some  programs  are  using  knowledge- 
based  practices  to  reduce  acquisition  risk,  implementation  continues  to  be 
inconsistent.  For  example,  programs  that  recently  began  system 
development  demonstrated  some  best  practices — such  as  completing  a 
system-level  preliminary  design  review  before  committing  resources. 
Others,  however,  are  carrying  technology  risk  well  into  system 
development,  failing  to  demonstrate  designs  through  prototyping,  or 
proceeding  into  production  before  ensuring  manufacturing  processes  are 
under  control  and  developmental  testing  is  complete. 


Our  body  of  work  has  shown  that  positive  acquisition  outcomes  require 
the  use  of  a  knowledge-based  approach  to  product  development  that 
demonstrates  high  levels  of  knowledge  attained  before  significant 
commitments  are  made.26  In  essence,  knowledge  supplants  risk  over 
time.  This  work  led  to  multiple  recommendations  that  DOD  has  generally 
or  partially  agreed  with  and  has  made  progress  in  implementing.  On  the 
basis  of  this  work,  we  identified  three  key  knowledge  or  decision  points 
during  the  acquisition  cycle — development  start,  system-level  critical 
design  review,  and  production  start — at  which  programs  need  to 
demonstrate  critical  levels  of  knowledge  to  proceed.  Figure  12  aligns  the 
acquisition  milestones  described  in  DOD’s  primary  acquisition  policy  with 
these  key  decision  points. 


26GAO,  Best  Practices:  DOD  Can  Achieve  Better  Outcomes  by  Standardizing  the  Way 
Manufacturing  Risks  Are  Managed,  GAO-10-439  (Washington,  D.C.:  Apr.  22,  2010);  Best 
Practices:  High  Levels  of  Knowledge  at  Key  Points  Differentiate  Commercial  Shipbuilding 
from  Navy  Shipbuilding,  GAO-09-322  (Washington,  D.C.:  May  13,  2009);  Defense 
Acquisitions:  A  Knowledge-Based  Funding  Approach  Could  Improve  Major  Weapon 
System  Program  Outcomes,  GAO-08-619  (Washington,  D.C.:  July  2,  2008);  Best 
Practices:  Capturing  Design  and  Manufacturing  Knowledge  Early  Improves  Acquisition 
Outcomes,  GAO-02-701  (Washington,  D.C.:  July  15,  2002);  Best  Practices:  Better 
Matching  of  Needs  and  Resources  Will  Lead  to  Better  Weapon  System  Outcomes, 

GAO-01 -288  (Washington,  D.C.:  Mar.  8,  2001);  and  Best  Practices:  Better  Management  of 
Technology  Development  Can  Improve  Weapon  System  Outcomes,  GAO/NSIAD-99-1 62 
(Washington,  D.C.:  July  30,  1999). 
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Figure  12:  Defense  Acquisition  Cycle  and  GAO  Knowledge  Points 
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•  Ninety  percent  of 

prototype  demonstrated  in 
intended  environment. 

•  Incremental  acquisition 
strategy  in  place 

engineering  drawings 
released 

•  Manufacturing  processes  in 
control 

•  Knowledge-based  cost 
estimate 

•  Early  integrated  system 
prototype  demonstrated 

•  Product  reliability 
demonstrated 

•  Critical  manufacturing 
process  identified 

Source:  GAO  analysis  of  DOD  provided  data,  DOD  Instruction  5000.02,  and  best  practices.  |  GAO-1 7-333SP 


As  our  prior  work  has  shown,  the  building  of  knowledge  consists  of 
information  that  should  be  gathered  at  these  three  critical  points  over  the 
course  of  a  program. 

Knowledge  point  1:  Resources  and  requirements  match.  Achieving  a 
high  level  of  technology  maturity  by  the  start  of  system  development  is 
one  of  several  important  indicators  of  whether  this  match  has  been  made 
This  means  that  the  technologies  needed  to  meet  essential  product 
requirements  should  be  form,  fit,  and  function,  integrated  with  other  key 
supporting  elements  and  have  been  demonstrated  to  work  in  a  realistic 
environment.  In  addition,  the  developer  should  complete  a  series  of 
systems  engineering  reviews  culminating  in  a  preliminary  design  of  the 
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product  that  shows  the  design  is  feasible.  Constraining  the  development 
phase  of  a  program  to  5  or  6  years  is  also  recommended  because  it 
aligns  with  DOD’s  budget  planning  process  and  fosters  the  negotiation  of 
trade-offs  in  requirements  and  technologies.  For  shipbuilding  programs, 
critical  technologies  should  be  matured  into  actual  sub-system  prototypes 
and  successfully  demonstrated  in  an  operational  environment  before  a 
contract  is  awarded  for  the  detail  design  of  a  new  ship. 

Knowledge  point  2:  Product  design  is  stable.  This  point  occurs  when  a 
program  determines  that  a  product’s  design  will  meet  customer 
requirements  as  well  as  cost,  schedule,  and  reliability  targets.  A  best 
practice  is  to  achieve  design  stability  at  the  system-level  critical  design 
review,  usually  held  midway  through  system  development.  Completion  of 
at  least  90  percent  of  engineering  drawings  at  this  point  provides  tangible 
evidence  that  the  product’s  design  is  stable,  and  a  prototype 
demonstration  shows  that  the  design  is  capable  of  meeting  performance 
requirements.  Shipbuilding  programs  should  demonstrate  design  stability 
by  completing  100  percent  of  the  basic  and  functional  drawings,  generally 
corresponding  to  a  three-dimensional  product  model,  by  the  start  of 
construction  for  a  new  ship.  Programs  can  also  improve  the  stability  of 
their  designs  by  conducting  reliability  growth  testing  and  completing 
failure  modes  and  effects  analyses  so  fixes  can  be  incorporated  before 
production  begins.  At  this  point,  programs  should  also  begin  preparing  for 
production  by  identifying  manufacturing  risks,  key  product  characteristics, 
and  critical  manufacturing  processes. 

Knowledge  point  3:  Manufacturing  processes  are  mature.  This  point 
is  achieved  when  it  has  been  demonstrated  that  the  developer  can 
manufacture  the  product  within  cost,  schedule,  and  quality  targets.  A  best 
practice  is  to  ensure  that  all  critical  manufacturing  processes  are  in 
statistical  control — that  is,  they  are  repeatable,  sustainable,  and  capable 
of  consistently  producing  parts  within  the  product’s  quality  tolerances  and 
standards — at  the  start  of  production.  Demonstrating  critical  processes  on 
a  pilot  production  line  is  an  important  initial  step  in  this  effort.  In  addition, 
production  and  post-production  costs  are  minimized  when  a  fully 
integrated,  capable  production-representative  prototype  is  demonstrated 
to  show  that  the  system  will  work  as  intended  in  a  reliable  manner  before 
committing  to  production.  We  did  not  assess  shipbuilding  programs  for 
this  knowledge  point  due  to  differences  in  the  production  processes  used 
to  construct  ships. 

Knowledge  in  these  three  areas  builds  over  time.  Our  prior  work  on 
knowledge-based  approaches  shows  that  a  knowledge  deficit  early  in  a 
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program  can  cascade  through  design  and  production,  leaving  decision 
makers  with  less  knowledge  to  support  decisions  about  when  and  how  to 
best  move  into  subsequent  acquisition  phases  that  commit  more 
budgetary  resources.  Demonstrating  technology  maturity  is  a  prerequisite 
for  moving  forward  into  system  development,  during  which  time  the  focus 
should  be  on  design  and  integration.  A  stable  and  mature  design  is  also  a 
prerequisite  for  moving  forward  into  production,  where  the  focus  should 
be  on  efficient  manufacturing.  Additional  details  about  key  practices  at 
each  of  the  knowledge  points  can  be  found  in  appendix  IV. 

For  this  report,  we  assessed  the  knowledge  attained  at  key  junctures  in 
the  acquisition  process  for  45  current  programs,  which  are  mostly  in 
development  or  early  production.27  Not  all  programs  included  in  our 
review  of  knowledge-based  practices  provided  information  for  every 
knowledge  point  and  some  had  not  reached  all  of  the  knowledge  points — 
development  start,  design  review,  and  production  start — at  the  time  of  this 
assessment.  We  also  reviewed  the  knowledge  that  9  future  major 
defense  acquisition  programs,  as  identified  by  DOD,  expect  to  attain 
when  they  start  system  development  in  the  coming  years.28  In  addition, 
we  analyzed  the  extent  to  which  programs  are  undertaking  concurrent 
developmental  testing  and  production  as  well  as  their  software 
development  efforts  to  determine  how  they  monitor  and  manage  them. 

Our  analysis  of  the  data  from  these  current  and  future  programs  allows  us 
to  make  the  following  five  observations. 


27Because  knowledge  points  and  best  practices  differ  for  shipbuilding  programs,  we 
exclude  the  six  shipbuilding  programs  in  DOD’s  portfolio  from  parts  of  our  analysis  at  each 
of  the  three  knowledge  points.  For  more  information,  see  appendix  I. 

28lnformation  for  these  programs  was  collected  from  two  data  collection  instruments 
distributed  to  program  officials.  See  the  “Analysis  of  Selected  DOD  Programs  Using 
Knowledge-Based  Criteria  and  Program  Concurrency”  section  of  appendix  I  for  more 
information. 
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Knowledge  Point  and  Program  Concurrency  Observations 

1 .  Of  the  four  programs  that  began  or  planned  to  begin  system  development  during 
our  assessment  period,  only  one  has  demonstrated  a  total  match  between 
resources  and  requirements.  One  program  plans  to  enter  system  development  with 
immature  technologies,  one  will  have  technologies  approaching  maturity,  and  two 
programs  have  not  identified  critical  technologies  to  evaluate.  Three  of  the 
programs  completed  a  preliminary  design  review  before  development  start.  Two 
did  not  conduct  all  of  the  early  system  engineering  reviews  recommended  by  best 
practices.  Two  of  four  programs  plan  to  constrain  their  development  to  less  than  6 
years. 

2.  Of  the  four  programs  that  held  or  planned  to  hold  a  critical  design  review  during  our 
assessment  period,  none  met  all  of  the  best  practices.  One  of  the  programs  has 
demonstrated  mature  technologies.  While  all  programs  plan  to  release  at  least  90 
percent  of  drawings,  three  also  do  not  plan  to  test  a  system-level  integrated 
prototype.  The  implementation  of  best  practices  by  these  programs  has  degraded 
almost  40  percent  from  what  we  observed  in  2016. 

3.  Of  the  three  programs  that  held  or  planned  to  hold  a  production  decision  during  our 
assessment  period,  none  met  all  of  the  best  practices.  Two  programs  did  not  test  a 
production-representative  prototype  before  making  a  production  decision,  and  only 
one  of  the  three  demonstrated  processes  on  a  pilot  production  line.  The 
implementation  of  best  practices  at  this  juncture  was  mixed  or  slightly  degraded 
compared  to  DOD  programs'  past  implementation. 

4.  Eighteen  programs  in  production  plan  to  complete  30  percent  or  more  of  their 
developmental  testing  concurrent  with  production.  Nine  of  these  18  programs  plan 
to  place  more  than  20  percent  of  their  procurement  quantities  under  contract 
before  testing  is  complete. 

5.  Of  the  54  current  and  future  programs  we  assessed,  41  reported  software 
development  as  a  high-risk  area.  Despite  this,  almost  half  of  the  current  programs 
do  not  separately  price  or  track  the  cost  of  the  software  development  efforts.  Of  the 
45  current  programs  we  assessed,  1 1  programs  plan  to  start  production  prior  to 
completing  software  development. 

Source:  GAO  analysis  of  DOD  data.  |  GAO-1 7-333SP 


1 .  Of  the  four  programs  that  began  or  planned  to  begin  system 
development  during  our  assessment  period,  only  one  has 
demonstrated  a  total  match  between  resources  and 
requirements.  One  program  plans  to  enter  system  development 
with  immature  technologies,  one  will  have  technologies 
approaching  maturity,  and  two  programs  have  not  identified 
critical  technologies  to  evaluate.  Three  of  the  programs 
completed  a  preliminary  design  review  before  development  start. 
Two  did  not  conduct  all  of  the  early  system  engineering  reviews 
recommended  by  best  practices.  Two  of  four  programs  plan  to 
constrain  their  development  to  less  than  6  years.  Our  prior  work 
shows  that  the  most  critical  juncture  in  any  major  defense  acquisition 
is  the  decision  to  start  system  development,  a  point  at  which 
knowledge-based  acquisition  practices  recommend  having  a  match 
between  what  DOD  wants  in  a  weapon  system,  as  defined  by  its 
requirements,  and  the  mature  technologies,  funding,  schedule,  and 
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other  resources  needed  to  develop  that  system.29  Figure  13  shows 
the  extent  to  which  DOD  has  implemented  recommended  acquisition 
practices  for  knowledge  point  1  for  the  four  programs  that  recently 
started  system  development  or  plan  to  do  so  in  early  2017:  F-15 
Eagle  Passive/Active  Warning  and  Survivability  System  (F-15 
EPAWSS),  Indirect  Fire  Protection  Capability  Increment  2-Intercept 
Block  1  (IFPC  Inc  2-1  Block  1),  the  Columbia  Class  Ballistic  Missile 
Submarine  (Columbia  Class  SSBN)  (formerly  the  Ohio  Class 
Replacement),  and  the  Presidential  Aircraft  Recapitalization  (PAR) — 
as  well  as  the  other  41  current  programs  we  assessed  that  previously 
accomplished  this  knowledge  point. 


29GAO,  Best  Practices:  Better  Matching  of  Needs  and  Resources  Will  Lead  to  Better 
Weapon  System  Outcomes ,  GAO-01-288  (Washington,  D.C.:  Mar.  8,  2001);  and  Best 
Practices:  Better  Management  of  Technology  Development  Can  Improve  Weapon  System 
Outcomes ,  GAO/NSIAD-99-162  (Washington,  D.C.:  July  30,  1999). 
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Figure  13:  Implementation  of  Knowledge-Based  Practices  for  Programs  in  System  Development 


Knowledge-based  practices 
at  system  development  start 

Demonstrate  all  critical  technologies  are  very 
close  to  final  form,  fit,  and  function  within  a 
relevant  environment  (TRL  6) 

Demonstrate  all  critical  technologies  are  in 
form,  fit,  and  function  within  an  operational 
environment  (TRL  7) 

Complete  system  functional  review  and 
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Columbia  Class  SSBN  =  Columbia  Class  Ballistic  Missile  Submarine 

F-15  EPAWSS  =  F-15  Eagle  Passive/Active  Warning  Survivability  System 

IFPC  Inc  2-1  Block  1=  Indirect  Fire  Protection  Capability  Increment  2-Intercept  Block  1 

PAR  =  Presidential  Aircraft  Recapitalization 

Source:  GAO  analysis  of  Department  of  Defense  data.  |  GAO-1 7-333SP 


Demonstrate  Technology  We  assessed  whether  programs  starting  system  development  met 

Maturity  two  different  best  practices  for  technology  maturity.  First,  federal 

statute  requires  that  programs  generally  obtain  a  certification  stating 
that  technologies  have  been  demonstrated  within  a  relevant 
environment  which  as  implemented  means  being  very  close  to  final 
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form,  fit,  and  function.30  Second,  knowledge-based  acquisition 
practices  recommend  that  programs  fully  mature  technologies  and 
demonstrate  them  in  an  operational — or  realistic — environment  prior 
to  starting  system  development.  This  is  a  higher  level  of  technology 
maturity  than  required  by  law.  Demonstrating  technologies  in  an 
operational  environment  is  a  better  indicator  of  whether  a  program 
has  achieved  a  resource  and  requirements  match,  as  it  demonstrates 
the  technologies’  form,  fit,  and  function  as  well  as  the  effect  of  the 
intended  environment  on  those  technologies. 

One  program — the  Columbia  Class  SSBN — does  not  plan  to 
demonstrate  all  of  its  critical  technologies  in  a  relevant  environment  at 
the  statutorily  required  level  and  has  obtained  a  waiver  to  do  so. 
Columbia  Class  SSBN  is  a  program  that  will  replace  the  Navy’s 
current  fleet  of  Ohio-class  ballistic  missile  submarines.  The  program 
began  system  development  in  January  2017  and  has  identified  two 
critical  technologies.  The  Navy  has  subsequently  determined  that  one 
is  no  longer  a  critical  technology  based  on  testing  in  a  relevant 
environment.  The  other  technology  was  assessed  to  be  immature,  but 
the  program  was  granted  a  waiver  from  meeting  this  requirement.  To 
mitigate  the  development  risk,  the  Navy  plans  to  share  parts  and 
components  with  the  Virginia-class  submarine,  to  the  extent  possible. 

We  also  found  that,  of  the  four  programs  that  began  or  were  planning 
to  begin  system  development  in  2016,  two  programs  had  not  fully 
matured  their  technologies  and  the  other  two  did  not  identify  any 
critical  technologies.  Fully  mature  technologies  are  defined  as  those 
that  have  been  demonstrated  in  their  final  form,  fit,  and  function  within 
an  operational  environment  with  a  prototype. 

An  independent  review  team,  led  by  the  Air  Force  Life  Cycle 
Management  Center  determined  that  the  F-15  EPAWSS  program’s 
technologies  were  approaching  maturity  at  the  start  of  system 
development,  but  were  not  fully  mature  by  best  practices.  The 
program  does  not  plan  on  attaining  full  maturity  until  after  its 


^Demonstration  in  a  relevant  environment  is  Technology  Readiness  Level  (TRL)  6. 
Demonstration  in  an  operational  environment  is  TRL  7.  See  appendix  V  for  detailed 
descriptions  of  TRLs.  In  addition,  a  major  defense  acquisition  program  generally  may  not 
receive  approval  for  development  start  until  the  milestone  decision  authority  certifies  that 
the  technology  in  the  program  has  been  demonstrated  in  a  relevant  environment.  10 
U.S.C.  §  2366b(a)(3)(D).  Under  certain  circumstances,  this  requirement  may  be  waived. 
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scheduled  critical  design  review  in  March  2017.  This  delay  leaves  it  at 
risk  for  flight  test  delays  if  the  technologies  do  not  mature  as  planned. 
The  Army  conducted  a  Technology  Readiness  Assessment  for  the 
IFPC  Inc  2-1  Block  1  program  that  identified  no  critical  technologies  at 
the  start  of  its  system  development.  This  is  due  to  the  maturity  of  the 
program’s  sub-systems,  which  have  already  been  fielded  or  are  in 
production.  Since  IFPC  Inc  2-1  Block  1  and  the  PAR  programs  do  not 
have  critical  technologies,  we  did  not  evaluate  them  for  this  best 
practice.  The  Columbia  Class  SSBN  technologies  are  immature  as 
the  program  begins  system  development,  as  the  Navy  has  not 
demonstrated  all  technologies  in  an  operational  environment.  As  our 
prior  work  on  shipbuilding  best  practices  has  shown,  this  strategy 
introduces  risk  for  cost  and  schedule  growth  in  the  program.31 

Of  the  remaining  41  programs,  just  3  reported  that  all  of  their  critical 
technologies  were  mature  to  best  practices — in  other  words, 
demonstrated  in  an  operational  environment — when  they  began 
development,  while  23  programs  reported  that  their  technologies  were 
nearing  maturity.  Among  all  programs  in  our  assessment,  just  7 
percent  began  system  development  with  fully  mature  technologies. 

DOD  acquisition  reforms  included  in  WSARA  and  DOD’s  “Better 
Buying  Power”  initiatives  are  intended  to  improve  acquisition 
procedures.  We  examined  whether  programs  started  before  or  after 
the  implementation  of  these  initiatives  met  the  best  practice  to  have 
mature  technologies  and  their  subsequent  levels  of  cost  growth.  To  do 
so,  we  assessed  (1)  whether  the  45  programs  started  development 
before  or  after  implementation  of  WSARA;  (2)  what  their  level  of 
technology  maturity  was  at  that  time;  and  (3)  their  development  cost 
growth.  Of  the  45  programs,  19  were  started  before  the 
implementation  of  WSARA,  25  were  started  after,  and  one  did  not 
have  a  system  development  start.  Nine  of  the  19  programs  that 
started  before  the  implementation  of  WSARA  began  development  with 
immature  technologies.  These  9  programs  include  3  shipbuilding 
programs —  CVN  78  Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier, 
DDG  1000  Zumwalt  Class  Destroyer,  and  Littoral  Combat  Ship — as 
well  as  the  CH-53K  Pleavy  Lift  Replacement  Helicopter,  the  F-35 
Lightning  II,  Joint  Tactical  Radio  System  Handheld,  Manpack,  and 


31  GAO,  Best  Practices:  High  Levels  of  Knowledge  at  Key  Points  Differentiate  Commercial 
Shipbuilding  from  Navy  Shipbuilding,  GAO-09-322  (Washington,  D.C.:  May  13,  2009). 
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Small  Form  Fit  Radios,  Littoral  Combat  Ship  Mission  Modules,  Patriot 
Advanced  Capability-3  Missile  Segment  Enhancement,  and  the 
Warfighter  Information  Network-Tactical  Increment  2  programs.  These 
programs  met  almost  none  of  the  development  start  best  practices 
recommended  in  our  body  of  work.  Since  they  began  system 
development,  as  a  group  they  have  realized  almost  $35  billion  in 
development  cost  growth.  In  contrast,  5  of  the  programs  that  started 
before  implementation  of  WSARA  began  development  with  nearly 
mature  or  fully  mature  technologies  and  have  realized  $1 .6  billion  in 
development  cost  growth,  a  significant  difference  from  that  of  the 
programs  with  immature  technologies  at  development  start.  The 
remaining  5  programs  did  not  report  on  the  technology  maturity  at  this 
point.  This  is  a  limited  set  of  programs  to  analyze,  but  we  have 
observed  that  those  using  fewer  best  practices  realized  significantly 
more  cost  growth.  As  a  result,  we  believe  that  programs  implementing 
more  of  the  best  practices  in  the  future  will  continue  to  realize  less 
cost  growth. 


Complete  Systems 
Engineering  Reviews 
before  Development  Start 


Knowledge-based  acquisition  practices  recommend  that  programs 
hold  systems  engineering  events,  including  a  system  requirements 
review  and  a  system  functional  review,  before  the  start  of  system 
development: 

•  System  requirements  reviews  ensure  that  requirements  have 
been  properly  identified  and  that  there  is  a  mutual  understanding 
between  the  government  and  the  contractor. 

•  System  functional  reviews  establish  a  baseline  for  the  planned 
system. 

We  found  that  2  programs  completed  these  reviews  prior  to  starting 
system  development  and  2  did  not.  The  Columbia  Class  SSBN 
program  held  a  system  requirements  review,  but  not  the  system 
functional  review.  The  PAR  program  held  neither  review. 

Regarding  the  use  of  this  best  practice  on  the  other  41  programs  we 
assessed,  37  percent  held  both  reviews  before  system  development 
start  and  61  percent  held  neither  or  only  one  of  these  reviews  before 
starting  system  development.  This  implementation  rate  is  similar  to 
last  year,  in  which  37  percent  of  the  programs  we  reviewed  held  both 
reviews  before  development  and  60  percent  held  neither. 
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Complete  Preliminary 
Design  Review  before 
System  Development  Start 


In  addition  to  the  engineering  reviews,  knowledge-based  acquisition 
practices  recommend  the  completion  of  a  preliminary  design  review, 
before  the  start  of  system  development  to  ensure  that  requirements 
are  defined  and  feasible,  and  that  the  proposed  design  can  meet 
those  requirements  within  cost,  schedule,  and  other  system 
constraints.  At  preliminary  design  review  is  when  a  program  should 
have  completed  sufficient  systems  engineering  so  as  to  establish  an 
“allocated  baseline” — essentially,  definition  of  all  subsystems  and  how 
they  are  to  work  together. 

In  a  November  2016  report,  we  found  that  establishing  a  preliminary 
design  through  early  detailed  systems  engineering  portends  better 
program  outcomes  than  doing  so  after  program  start.32  WSARA 
emphasized  the  value  of  defining  an  allocated  baseline  early  by 
establishing  a  statutory  requirement  to  conduct  a  preliminary  design 
review  prior  to  starting  a  development  program  or  obtain  a  waiver  to 
the  requirement.33  Among  the  4  programs  that  started  system 
development  during  our  assessment  period,  all  but  one  of  them 
conducted  this  review  before  beginning  system  development.  PAR 
obtained  a  waiver  to  this  requirement  and  plans  to  hold  its  preliminary 
design  review  in  April  201 8,  over  a  year  and  a  half  after  its 
development  start.  Nineteen  of  the  other  41  programs  held  a 
preliminary  design  review  before  the  start  of  system  development. 
Sixty-eight  percent  of  programs  that  began  development  since  the 
implementation  of  WSARA  are  implementing  this  practice  compared 
to  26  percent  of  programs  that  began  development  before  the 
implementation  of  WSARA,  which  is  an  improvement  from  last  year’s 
assessment. 


Begin  Future  Programs  As  part  of  our  analysis,  we  assessed  9  future  programs  scheduled  to 

with  Adequate  Knowledge  become  major  defense  acquisition  programs  in  coming  years.  These 


32GAO,  Weapon  System  Requirements:  Detailed  Systems  Engineering  Prior  to  Product 
Development  Positions  Programs  for  Success,  GAO-17-77  (Washington,  D.C.:  Nov.  17, 
2016). 

33Pub.  L.  No.  1 11-23,  §  205(a).  A  major  defense  acquisition  program  may  not  receive 
milestone  B  approval  until  the  program  has  held  a  preliminary  design  review  and  the 
milestone  decision  authority  has  conducted  a  formal  post-preliminary  design  review 
assessment  and  certified  on  the  basis  of  such  assessment  that  the  program  demonstrates 
a  high  likelihood  of  accomplishing  its  intended  mission  unless  a  waiver  is  properly  granted 
by  the  milestone  decision  authority.  1 0  U.S.C.  §  2366b(a)(2),  (d)(1). 
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programs  provided  information  on  the  knowledge  they  planned  to 
obtain  and  the  best  practices  they  intend  to  implement  before  their 
system  development  start  is  approved.  Four  identified  critical 
technologies,  2  of  which  identified  their  anticipated  maturity  levels 
expected  at  system  development  start.  One  future  program — the 
T-AO  205  John  Lewis  Class  Fleet  Oiler  (formerly  T-AO(X)) — reported 
its  critical  technologies  as  fully  mature  in  October  2014,  almost  3 
years  before  its  planned  development  start.  Two  of  the  future 
programs  reported  that  their  critical  technologies  are  expected  to  be 
nearing  maturity  and  the  remaining  program  reported  that  its  critical 
technology  maturity  has  not  yet  been  determined  for  the  time  of  their 
system  development  start. 

Unlike  the  programs  that  held  system  development  start  in  the  past 
year,  only  2  of  the  9  future  programs  plan  to  hold  a  preliminary  design 
review  before  the  start  of  system  development  and  only  3  programs 
plan  to  conduct  both  a  system  functional  and  system  requirements 
review  before  that  time.  While  all  but  one  of  the  future  programs 
currently  plan  to  limit  their  system  development  phase  to  6  years  or 
less,  these  plans  are  preliminary  and  the  programs  are  at  risk  of  not 
satisfying  all  the  knowledge-based  practices  we  reviewed,  leaving 
them  at  risk  for  cost  and  schedule  growth,  as  table  5  indicates.  The 
remaining  programs  reported  that  their  critical  technologies  would  not 
be  fully  mature  or  have  not  been  identified.  One  program’s  technology 
maturity  will  be  determined  at  system  development  start. 


Table  5:  Projected  Implementation  of  Knowledge-Based  Practices  for  Future  Programs 


Development 

start 

Projected  to 
demonstrate  all 
critical  technologies  in 
an  operational 
environment 

Projected  to 
complete  all 
systems 
engineering 
reviews 

Plan  to 
constrain 
system 
development 

Long  Range  Precision  Fires 

TBD 

— 

O 

• 

T-AO  205  John  Lewis  Class  Fleet  Oiler 

06/2017 

• 

• 

• 

P-8A  Poseidon  Multi-Mission  Maritime  Aircraft 
Increment  3 

NA 

— 

0 

— 

MQ-25  Stingray  Unmanned  Air  System 

05/2018 

— 

0 

• 

Joint  Surveillance  Target  Attack  Radar  System 
Recapitalization 

10/2017 

— 

0 

• 

Improved  Turbine  Engine  Program 

TBD 

O 

0 

• 

Amphibious  Ship  Replacement 

TBDa 

— 

0 

• 

Advanced  Pilot  Training 

12/2017 

— 

0 

• 
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Development 

start 

Projected  to 
demonstrate  all 
critical  technologies  in 
an  operational 
environment 

Projected  to 
complete  all 
systems 
engineering 
reviews 

Plan  to 
constrain 
system 
development 

Weather  Satellite  Follow-On 

06/2018 

O 

• 

• 

•  Implementation  planned 
O  No  implementation  planned 
—  Practice  to  be  determined 

Source:  GAO  analysis  of  DOD  data.  |  GAO-1 7-333SP 

Note:  We  assessed  the  P-8A  Poseidon  Multi-Mission  Maritime  Aircraft  Increment  3  as  part  of  the  9 
future  major  defense  acquisition  programs  not  yet  in  the  portfolio.  During  the  course  of  our  review,  we 
learned  that  DOD  no  longer  plans  to  manage  Increment  3  separately  from  the  existing  P-8A  program. 
We  will  reflect  this  change  in  future  assessments  of  the  P-8A  program. 

aThe  Amphibious  Ship  Replacement  is  by-passing  system  development  and  entering  the  DOD 
acquisition  process  in  the  production  phase  at  a  point  which  is  to  be  determined. 


Enforcing  discipline  and  accountability  at  the  start  of  development  are 
essential  for  establishing  satisfactory  acquisition  outcomes.  Top 
decision  makers  need  to  ensure  that  programs  will  meet  a  knowledge- 
based  approach  before  the  programs  are  approved  and  funded. 
However,  our  observations  show  that  the  future  programs  plan  to 
proceed  without  meeting  these  best  practices  which  creates  an 
environment  where  they  are  not  held  to  DOD’s  standard,  which  makes 
successful  acquisition  outcomes  unlikely. 

2.  Of  the  four  programs  that  held  or  are  planning  to  hold  a  critical 
design  review  during  our  assessment  period,  none  met  all  of  the 
best  practices.  One  of  the  programs  has  demonstrated  mature 
technologies.  While  all  programs  plan  to  release  at  least  90 
percent  of  drawings,  three  also  do  not  plan  to  test  a  system-level 
integrated  prototype.  The  implementation  of  best  practices  by 
these  programs  has  degraded  almost  40  percent  from  what  we 
observed  in  2016.  For  this  key  milestone  in  a  program’s  acquisition, 
we  assessed  eight  best  practices  to  determine  the  extent  to  which 
programs  are  attaining  the  knowledge  needed  for  a  low-risk 
acquisition.  We  have  previously  reported  that  programs  that  hold  their 
critical  design  review  before  achieving  knowledge  of  a  stable 
demonstrated  design  also  experience  higher  average  costs  and 
longer  schedule  delays.  Figure  14  shows  the  extent  to  which 
recommended  acquisition  practices  for  knowledge  point  2  have  been 
implemented  for  the  four  programs  that  recently  finished  their  critical 
design  review,  or  plan  to  in  early  2017:  Armored  Multi-Purpose 
Vehicle  (AMPV),  Common  Infrared  Countermeasure  (CIRCM), 
Offensive  Anti-Surface  Warfare  Increment  1  (OASuW  Inc  1),  VH-92A 
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Presidential  Helicopter  Replacement  Program  (VH-92A) — as  well  as 
the  other  31  current  programs  we  assessed,  which  previously 
accomplished  this  knowledge  point. 


Figure  14:  Implementation  of  Knowledge-Based  Practices  for  Selected  Programs  at  Critical  Design  Review 


7  k  /  /  Other  31 

/  /  /  Pro9rams 


Knowledge-based  practices 
at  critical  design  review 

// 

'  /////// 
/cj/cy  /.£•/• 

/o 

/- 

Demonstrate  all  critical  technologies  in  an 
operational  environment 

• 

O 

o 

— 

6 

19 

6 

Release  at  least  90  percent  of  drawings 

• 

• 

• 

• 

10 

16 

5 

Test  an  early  system-level  integrated 
prototype 

o 

• 

o 

o 

5 

19 

7 

Establish  a  reliability  growth  curve 

• 

• 

— 

• 

21 

6 

4 

Identify  key  product  characteristics 

• 

— 

• 

• 

27 

0 

4 

Identify  critical  manufacturing  processes 

• 

— 

• 

• 

25 

1 

5 

Conduct  producibility  assessments  to  identify 
manufacturing  risks  for  key  technologies 

• 

— 

• 

— 

24 

2 

5 

Complete  failure  modes  and  effects  analysis 

• 

• 

• 

• 

24 

1 

6 

O 


Practice  implemented 
Practice  not  implemented 


- Practice  not  applicable  or  information  not  available  per  the  program  office  response 


AMPV  =  Armored  Multi-Purpose  Vehicle 

CIRCM  =  Common  Infrared  Countermeasure 

OASuW  Inc  1  =  Offensive  Anti-Surface  Warfare  Increment  1 

VH-92A  =  VH-92A  Presidential  Helicopter  Replacement  Program 

Source:  GAO  analysis  of  Department  of  Defense  data.  |  GAO-17-333SP 


Demonstrate  Technology 
Maturity 


As  product  knowledge  is  cumulative,  by  the  critical  design  review, 
programs  should  have  demonstrated  critical  technologies  in  an 
operational  environment  to  ensure  that  the  product  can  meet 
requirements.  Failure  to  fully  mature  technologies  prior  to  developing 
the  system  design  can  lead  to  redesign  and  cost  and  schedule  growth 
if  later  discoveries  during  development  lead  to  revisions.  Of  all  the 
knowledge-based  acquisition  practices  programs  strive  to  implement 
throughout  the  acquisition  life  cycle,  this  is  one  of  the  least  utilized. 
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The  OASuW  Inc  1  program  does  not  plan  to  meet  this  best  practice 
before  completing  its  critical  design  review.  This  program  has  six 
critical  technologies;  five  are  immature  and  one  is  approaching 
maturity,  but  none  will  be  demonstrated  in  an  operational  environment 
at  the  time  of  the  program’s  critical  design  review.  Additionally,  the 
CIRCM  program  identified  seven  critical  technologies,  which  were  not 
mature  at  the  program’s  critical  design  review.  The  AMPV  program  is 
the  only  program  with  mature  technologies  that  held  a  critical  design 
review  this  year.  The  VH-92A  program  did  not  identify  critical 
technologies  at  this  juncture;  therefore,  we  did  not  evaluate  it  for  that 
practice. 

To  assess  the  effect  of  technology  maturity  on  cost  over  time,  we  also 
examined  the  19  programs  that  started  development  7  or  more  years 
ago  and  the  level  of  their  technology  maturity  at  the  time  of  their 
critical  design  review.  We  found  that  of  these  19  programs,  6  held 
their  critical  design  review  with  immature  technologies.  These  6 
programs  have  realized  over  $33  billion,  or  63  percent,  in 
development  cost  growth  over  their  initial  cost  estimates.  These 
include  the  same  three  shipbuilding  programs  mentioned  in  the 
previous  section — CVN  78  Gerald  R.  Ford  Class  Nuclear  Aircraft 
Carrier,  Littoral  Combat  Ship,  and  DDG  1000  Zumwalt  Class 
Destroyer — which  started  system  development  with  immature 
technologies.  They  also  include  the  F-35  Lightning  II,  M109A7  Family 
of  Vehicles,  and  Patriot  Advanced  Capability-3  Missile  Segment 
Enhancement  programs.  In  contrast,  8  programs  had  fully  mature 
technologies  or  nearly  mature  technologies  at  their  critical  design 
review.  These  8  programs  realized  $4.7  billion,  or  31  percent,  in 
development  cost  growth,  which  is  significantly  less  than  the 
programs  with  immature  technologies. 


Demonstrate  System  All  of  the  programs  holding  their  critical  design  review  in  2016 

Design  Stability  demonstrated  design  stability  by  releasing  over  90  percent  of  their 

expected  design  drawings.  This  is  an  encouraging  development,  as  it 
represents  the  second  year  in  a  row  in  which  all  programs  being 
evaluated  met  this  best  practice.  Out  of  the  other  31  programs  that 
previously  held  their  design  reviews,  1 0  met  this  best  practice  and  1 6 
did  not.  For  the  5  remaining  programs,  the  program  offices  reported 
this  practice  was  not  applicable. 

Testing  of  an  early  prototype  is  useful  for  demonstrating  that  a  system 
will  work  as  intended  and  can  be  built  within  cost  and  schedule.  Only 
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1  of  the  4  programs — CIRCM — tested  a  system-level  integrated 
prototype  prior  to  its  critical  design  review.  This  is  a  decline  since  last 
year’s  review,  when  only  2  of  the  5  programs  we  evaluated  did  not 
test  a  system-level  prototype  before  their  design  review. 

One  program  that  did  not  test  a  system-level  integrated  prototype  is 
the  VH-92A  program,  which  is  developing  a  helicopter  for  transporting 
the  President,  Vice  President,  heads  of  state,  and  other  dignitaries. 
The  program  entered  system  development  in  2014  and  conducted  its 
design  review  in  July  2016.  VH-92A  does  not  have  any  critical 
technologies,  and  it  plans  to  reduce  risk  by  utilizing  a  commercial 
aircraft  and  mature  technologies,  which  are  already  in  use.  The 
program  demonstrated  some  design  stability  by  completing  all  of  its 
design  drawings,  but  it  did  not  test  a  system-level  integrated 
prototype.  Instead,  it  plans  to  verify  its  design  by  testing  a  fully 
configured,  production-level  prototype  in  2018.  Additionally,  while  both 
the  AMPV  and  OASuW  Inc  1  programs  demonstrated  some  design 
stability  by  releasing  more  than  90  percent  of  their  design  drawings, 
neither  tested  a  system-level  prototype  before  critical  design  review. 
The  AMPV  is  a  derivative  vehicle  utilizing  mature  technologies  and 
sub-systems  and  the  program’s  acquisition  strategy  did  not  allow  time 
to  develop  and  test  an  early  system-level  prototype.  AMPV  plans  to 
test  a  system-level  prototype  in  the  future.  OASuW  Inc  1  plans  to  test 
its  system-level  prototype  in  March  2017. 

We  assessed  31  other  programs  that  held  a  critical  design  review 
prior  to  2016,  and  found  that  only  5  tested  an  early  integrated 
prototype  before  critical  design  review.  We  did  not  assess  shipbuilding 
programs  against  this  knowledge-based  acquisition  practice  as  testing 
early  system-level  prototypes  in  these  programs  may  sometimes  be 
impractical,  particularly  in  instances  when  these  programs  are  not 
delivering  large  quantities.  The  limited  use  of  this  testing  before 
design  review  among  the  programs  we  assessed  shows  no 
improvement  from  our  prior  assessments.  Of  all  the  knowledge-based 
acquisition  practices  programs  strive  to  implement  throughout  the 
acquisition  life  cycle,  this  is  the  least  utilized  at  this  juncture.  Without 
demonstrating  at  this  point  that  the  system  can  work  as  intended,  the 
risk  that  the  program  will  face  design  changes  and  schedule  delays 
later  in  the  acquisition  cycle  increases. 
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Remaining  Critical  Design 
Review  Best  Practices 


Of  the  4  programs  we  assessed  only  the  AMPV  plans  to  implement 
the  last  five  knowledge-based  practices  shown  in  figure  14  to  increase 
confidence  in  the  stability  of  the  product’s  design  and  its  effect  on 
production.  This  includes  establishing  a  reliability  growth  curve, 
identifying  key  product  characteristics,  identifying  critical 
manufacturing  processes,  conducting  producibility  assessments,  and 
completing  a  failure  mode  and  effects  analysis.  The  exceptions  come 
from  three  programs  that  are  not  implementing  one  or  more  of  these 
practices  and  instead  report  the  practices  are  not  applicable  for 
various  reasons,  which  include  limited  production  quantities  and  lack 
of  identified  critical  technologies.  For  the  other  31  programs  in  this 
assessment  that  have  already  held  their  critical  design  review,  a 
majority  of  them  reported  using  each  of  these  practices. 

3.  Of  the  three  programs  that  held  a  production  decision  during  our 
assessment  period,  none  met  all  of  the  best  practices.  Two 
programs  did  not  test  a  production-representative  prototype 
before  making  a  production  decision,  and  only  one  program 
demonstrated  processes  on  a  pilot  production  line.  The 
implementation  of  best  practices  at  this  juncture  was  mixed  or 
slightly  degraded  compared  to  DOD  programs’  past 
implementation.  Capturing  critical  manufacturing  knowledge  before 
entering  production  ensures  that  a  weapon  system  will  work  as 
intended  and  can  be  manufactured  efficiently  to  meet  cost,  schedule, 
and  quality  targets.  This  knowledge  can  be  captured  through  the  use 
of  various  proactive  methods,  including  the  use  of  statistical  process 
control  data,  pilot  production  lines,  manufacturing  readiness  levels, 
and  prototype  testing.  There  are  three  programs  that  held  a 
production  decision  during  our  assessment  period-the  F-22  Increment 
3.2B  Modernization  (F-22  Inc  3.2B),  the  KC-46A  Tanker 
Modernization  (KC-46A),  and  the  Offensive  Anti-Surface  Warfare 
Increment  1  (OASuW  Inc  1).  Figure  15  shows  the  extent  to  which 
these  programs  have  implemented  the  associated  knowledge-based 
practices. 
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^  Practice  implemented 

O  Practice  not  implemented 

- Practice  not  applicable  or  information  not  available  per  the  program  office  response 

F-22  Inc  3.2B  =  F-22  Increment  3.2B  Modernization 
KC-46A=  KC-46A  Tanker  Modernization  Program 
OASuW  Inc  1  =  Offensive  Anti-Surface  Warfare  Increment  1 
Source:  GAO  analysis  of  Department  of  Defense  data.  |  GAO-1 7-333SP 

bWe  have  excluded  shipbuilding  programs  from  this  analysis  due  to  differences  in  the  production 
processes  used  to  construct  ships. 


Demonstrate 
Manufacturing  Process 
Capabilities  Are  in  Control 


None  of  the  programs  that  held  recent  production  decisions 
demonstrated  that  their  manufacturing  process  capabilities  were  in 
control.  Our  prior  work  has  shown  that  capturing  critical  manufacturing 
knowledge  before  entering  production  helps  ensure  that  the  system 
will  work  as  intended  and  will  meet  cost,  schedule,  and  quality  targets. 
Of  the  remaining  15  non-shipbuilding  programs  we  assessed,  which 
held  production  decisions  before  our  current  assessment  period,  one 
provided  data  indicating  that  critical  manufacturing  processes  were  in 
control  at  the  time  of  their  production  start.  Of  all  the  knowledge- 
based  acquisition  practices  described,  this  is  the  least  utilized  at  this 
juncture. 
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Demonstrate  Critical 
Processes  on  a  Pilot 
Production  Line 


One  of  the  programs  that  held  a  production  decision  over  the  past 
year  demonstrated  critical  processes  on  a  pilot  production  line.  This  is 
a  fair  decline  from  what  we  observed  last  year,  when  all  5  of  the 
programs  we  reviewed  met  this  knowledge  practice.  Of  the  21 
programs  we  assessed  that  plan  to  hold  a  production  decision  in  the 
future,  1 1  indicated  that  they  intended  to  test  a  pilot  production  line 
before  production  start.  Of  the  remaining  15  non-shipbuilding 
programs,  8  have  demonstrated  this  best  practice  whereas  4  did  not. 


Our  body  of  work  has  shown  that  production  and  post-production 
costs  are  also  minimized  when  a  fully  integrated,  production- 
representative  prototype  is  demonstrated  prior  to  the  production 
decision,  as  making  design  changes  after  production  begins  can  be 
both  costly  and  inefficient.  The  KC-46A  plans  to  meet  this  best 
practice,  but  the  F-22  Inc  3.2B  and  OASuW  Inc  1  do  not  plan  to 
complete  this  testing  until  approximately  4  months  and  1  year, 
respectively,  after  their  production  decision  dates.  Additionally,  16  of 
the  programs  we  assessed  that  plan  to  hold  their  production  decision 
in  the  future  intend  to  test  a  fully  configured  prototype.  Programs  that 
do  not  conduct  testing  of  production-representative  prototypes  prior  to 
beginning  manufacturing  risk  discovering  deficiencies  late  in  testing, 
which  may  trigger  the  need  for  expensive  re-tooling  of  production  lines 
and  retrofitting  of  articles  that  have  completed  production.  Of  the  other 
15  non-shipbuilding  programs  we  assessed,  6  reported  testing  a 
production-representative  prototype  before  this  decision. 

4.  Eighteen  programs  in  production  plan  to  complete  30  percent  or 
more  of  their  developmental  testing  concurrent  with  production. 
Nine  of  these  18  programs  plan  to  place  20  percent  or  more  of 
their  procurement  quantities  under  contract  before  testing  is 
complete.  Conducting  developmental  testing  while  in  production 
increases  the  risk  of  finding  problems  that  could  lead  to  design 
changes  and  costly  retrofits.  Developmental  testing  is  intended  to 
demonstrate  that  a  chosen  design  has  the  capabilities  required  and  to 
discover  and  fix  problems  before  a  system  enters  production.  Though 
DOD  policy  allows  some  degree  of  concurrency  between  initial 
production  and  developmental  testing,  beginning  production  before 
demonstrating  that  a  system  will  work  as  intended  increases  the  risk 
of  deficiencies  that  require  substantial  design  changes  and  costly 
modifications  to  already-produced  systems. 

Eighteen  of  the  programs  we  assessed  plan  to  complete  a  significant 
portion  of  their  testing — 30  percent  or  more — after  they  have  already 


Test  a  Production- 
representative  Prototype 
in  Its  Intended 
Environment 
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started  production.  Further,  9  of  these  programs  also  plan  to  put  20 
percent  or  more  of  their  procurement  quantities  under  contract  before 
the  testing  in  complete.  These  programs  are  the  DDG  1000  Zumwalt 
Class  Destroyer,  F-22  Increment  3.2B  Modernization,  Global 
Positioning  System  III,  LHA  6  America  Class  Amphibious  Assault 
Ship,  M109A7  Family  of  Vehicles,  MQ-8  Fire  Scout  Unmanned 
Aircraft  System,  Offensive  Anti-Surface  Warfare  Increment  1,  F-35 
Lightning  II,  and  the  Family  of  Advanced  Beyond  Line-of-Sight 
Terminals.  Concurrency  is  also  justified  to  expedite  the  development 
of  an  accelerated  acquisition  program.  However,  concurrently 
developing  and  producing  high  cost  systems  without  knowing  whether 
their  capabilities  can  be  demonstrated  is  a  high-risk  strategy. 

5.  Of  the  54  current  and  future  programs  we  assessed,  41  reported 
software  development  as  a  high-risk  area.  Despite  this,  almost 
half  of  the  current  programs  do  not  separately  price  or  track  the 
cost  of  the  software  development  efforts.  Of  the  45  current 
programs  we  assessed  1 1  plan  to  start  production  prior  to 
completing  software  development.  Software  development  has 
similar  phases  to  that  of  hardware  and — in  the  case  of  new  systems — 
occurs  in  parallel  with  hardware  development  until  software  and 
hardware  components  are  integrated.  According  to  DOD  policy,  major 
contracts  and  subcontracts  for  contractors  developing  or  producing 
software  elements  for  major  defense  acquisition  programs  are 
required  to  submit  a  report,  which  includes  software  resources  if  the 
effort  is  projected  to  be  greater  than  $20  million.34  Half  of  the 
programs  we  assessed  do  not  track  the  cost  of  their  software 
development,  citing  a  variety  of  reasons.  For  example,  several 
programs  reported  that  the  software  is  embedded  into  the  overall 
system  development  and  would  be  difficult  to  extract. 

Nonetheless,  of  the  45  current  programs  we  surveyed,  35  indicated 
that  they  had  identified  software  development  as  a  high-risk  area, 
similar  to  the  number  of  programs  that  reported  this  in  2016.  Of  the  9 
future  programs,  6  identified  their  software  development  as  high  risk. 

Programs  report  a  variety  of  reasons  that  lead  to  characterizing  their 
software  development  effort  as  high  risk.  Common  reasons  include: 


34Specifically,  all  major  contracts  and  subcontracts,  regardless  of  contract  type,  for 
contractors  developing  or  producing  software  elements  within  MDAP  and  pre-MDAP 
programs,  as  well  as  other  certain  acquisitions,  such  as  those  of  certain  automated 
information  systems,  for  any  software  development  element  with  a  projected  software 
effort  greater  than  $20  million  (then-year  dollars)  are  required  to  complete  a  Software 
Resources  Data  Report.  DOD  Instruction  5000.02. 
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•  Completion  of  the  originally  planned  software  effort  has  proved  to 
be  more  difficult  than  expected  (27  programs), 

•  Completion  of  the  software  effort  needed  to  conduct 
developmental  testing  successfully  (30  programs), 

•  Changes  to  meet  cyber  security  needs  led  to  additional  software 
development  efforts  (25  programs),  and 

•  Completion  of  the  software  effort  to  conduct  operational  testing 
successfully  (21  programs). 

Of  the  35  programs  identifying  software  development  as  high  risk,  26 
percent  are  not  tracking  their  software  costs.  Programs  not  tracking 
software  costs  limit  the  information  managers  and  decision  makers 
have,  making  it  difficult  to  manage  and  oversee  these  programs.  DOD 
Instruction  5000.02  provides  programs  several  types  of  acquisition 
models  from  which  to  choose.  Two  of  the  54  programs  in  our 
assessment  reported  using  the  defense  unique  software  intensive 
program  model,  and  3  identified  themselves  as  “hybrid  program — 
software  dominant.”  A  majority  of  programs  estimated  that  between  0 
to  20  percent  of  their  total  acquisition  cost  is  for  software. 

Eleven  of  the  45  current  programs  we  assessed  plan  to  begin 
production  prior  to  completing  the  software  development  necessary 
for  integration  with  system  hardware  and  achieving  baseline 
capabilities.  DOD  policy  allows  for  some  degree  of  concurrency 
between  initial  production  and  the  completion  of  developmental 
testing,  especially  for  the  completion  of  software.  While  some 
concurrency  may  be  necessary  when  rapidly  fielding  urgently  needed 
capabilities,  pursuing  software  development  while  the  system  is  in 
production  may  introduce  risks  if  problems  are  discovered  late  in 
testing. 


Assessments  of 
Individual  Programs 


This  section  contains  assessments  of  individual  weapon  programs 
grouped  by  lead  service — Army,  Navy  and  Marine  Corps,  Air  Force,  and 
DOD-led — and  includes  a  lead  service  separator  page  at  the  start  of  each 
grouping.  Each  assessment  presents  data  on  the  extent  to  which 
programs  are  following  a  knowledge-based  acquisition  approach  to 
product  development,  and  other  program  information.  Each  lead  service 
separator  page  summarizes  information  about  the  acquisition  phase, 
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current  estimated  funding  needs,  cost  and  schedule  growth,  and  product 
knowledge  attained.  In  total,  we  present  information  on  54  programs.35 

For  43  programs,  we  produced  two-page  assessments  discussing  the 
technology,  design,  and  manufacturing  knowledge  obtained,  as  well  as 
other  program  issues.  Each  two-page  assessment  also  contains  a 
comparison  of  total  acquisition  cost  from  the  first  full  estimate  for  the 
program  to  the  current  estimate.  The  first  full  estimate  is  generally  the 
cost  estimate  established  at  development  start;  however,  for  a  few 
programs  that  did  not  have  such  an  estimate,  we  used  the  estimate  at 
production  start  instead.  For  shipbuilding  programs,  we  used  their 
planning  estimates  if  those  estimates  were  available.  For  programs  that 
began  as  non-major  defense  acquisition  programs,  we  used  the  first  full 
estimate  available.  All  of  these  43  two-page  assessments  are  of  major 
defense  acquisition  programs,  most  of  which  are  in  development  or  early 
production  and  3  assessments  are  of  programs  that  were  projected  to 
become  major  defense  acquisition  programs  during  or  soon  after  our 
review.  See  figure  16  for  an  illustration  of  the  layout  of  each  two-page 
assessment. 

In  addition,  we  produced  one-page  assessments  on  the  current  status  of 
1 1  programs,  which  include  9  future  major  defense  acquisition  programs 
and  2  major  defense  acquisition  programs  that  are  well  into  production, 
but  are  developing  new  increments  of  capability  as  part  of  their  existing 
programs.  We  also  produced  a  one-page  assessment  of  the  Navy’s 
Frigate  initiative,  which  DOD  has  not  yet  formally  identified  as  a  future 
major  defense  acquisition  program. 


35ln  addition  to  the  54  program  assessments,  we  separately  assessed  the  Navy's  Frigate 
initiative,  which  is  currently  part  of  the  Navy’s  Littoral  Combat  Ship  program. 
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Figure  16:  Illustration  of  Program  Two-Page  Assessment 
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Airborne  and  Maritime/Fixed  Station  (AMF) 


The  Army's  AMF  program  plans  to  acquire  non- 
developmental,  software-defined  radios — the  Small 
Airborne  Networking  Radio  (SANR)— and 
associated  equipment  for  integration  Into  Army 
rotary  wing  and  unmanned  aerial  systems  These 
radios  will  provide  simultaneous  voice  and  data 
communications  between  Army  platforms  and 
ground  forces.  The  program  previously  planned  to 
also  acquire  the  Small  Airborne  Link  16  Terminal 
(SALT)  radio,  but.  in  August  201 5.  the  Army  directed 
program  officials  to  close  out  the  SALT  sub¬ 
program.  We  assessed  SANR  only. 


© 


In  July  201 2,  as  part  of  an  overall  Joint  Tactical 
Radio  System  (JTRS)  reorganization,  DOO 
directed  the  AMF  program  to  pursue  a 
restructured  acquisition  approach  and  acquire  the 
desired  radios  as  a  modified  non-developmental 
Hem,  leveraging  to  the  maximum  extent  practical 
investments  made  since  2008  within  the  original 
program  As  a  result,  the  program  will  procure 
existing  radios,  which  will  be  tested  for  technology 
maturity  as  part  of  the  formal  testing  process  Due 
to  the  program's  non-developmental  strategy,  we 
do  not  have  insight  Into  the  percentage  of  design 
drawings  released  and  officials  report  the  testing 
of  a  system-level  integrated  prototype  is  not 
applicable.  The  first  planned  procurement  of 
radios  is  currently  scheduled  for  after  the  non- 
developmental  item  contract  award,  which  is 
expected  in  2019  with  full-rate  production 
decision  scheduled  to  begin  in  2023 


Attainment  ot  Product  Knowt«dg> 


Common  Name:  AMF 


AMF  Program  ^  y 


Technology,  Design,  and  Production  Maturity 
In  July  2012— as  part  of  an  overall  JTRS 
reorganization  of  several  related  programs — the 
Under  Secretary  of  Defense  for  Acquisition, 
Technology  and  Logistics  directed  the  AMF  program 
to  pursue  a  restructured  acquisition  approach  and 
acquire  the  desired  radios  as  a  modified  non- 
developmental  item,  leveraging  to  the  maximum 
extent  practical  prior  investments  made  on  the 
original  program  since  development  started  in  2008 
Under  the  proyam's  original  acquisition  strategy, 
AMF  achieved  a  stable  design  by  releasing  at  least 
90  percent  of  its  drawings  and  testing  an  integrated 
prototype  before  it  held  its  critical  design  review 
The  restructuring  of  the  acquisition  strategy, 
however,  shifted  the  program  from  a  development 
effort  supporting  Army,  Air  Force,  and  Navy 
platforms  to  a  non-developmental  effort  that 
supports  only  Army  aviation  efforts  Since  the 
government  is  procuring  an  already  existing  item 
from  a  commercial  entity  in  a  non-developmental 
effort,  the  design  knowledge  criteria  related  to 
drawings  and  prototypes  are  no  longer  applicable 
The  program  officials  have  identified  critical 
technologies  necessary  for  the  existing  radios  the 
Army  intends  to  procure,  and  the  program  plans  to 
have  the  technology  maturity  demonstrated  as  part 
of  the  overall  test  and  demonstration  process.  The 
program  does  not  intend  to  develop  any  new 
technologies  or  software  for  the  radios  In  2014,  the 
Army  split  AMF  into  two  separate  sub  programs— 
SALT  and  SANR  In  August  201 5,  the  Army  directed 
the  close-out  of  the  SALT  sub  program  as  these 
radios  would  not  have  met  the  Army's  operational 
requirements  until  fiscal  year  2021  and  were 
expected  to  be  more  expensive  than  other  options. 
The  Army  no  longer  intends  to  procure  these  radios. 

The  SANR  sub-program  is  currently  in  the  pre¬ 
solicitation  phase  No  production  contracts  have 
been  awarded  yet.  Program  officials  stated  that  the 
non-developmental  Item  strategy  will  ensure  that  a 
certain  level  of  production  readiness  is  achieved 
They  added  that  110  radios,  out  of  the  7,111 
expected,  will  be  purchased  for  verification  testing 
and  initial  platform  integration.  Reliability  verification 
testing  is  expected  to  begin  after  purchase  of  the 
first  radios  in  fiscal  year  2019  and  will  be  completed 
prior  to  the  start  of  fuH-rate  production  in  2023. 


The  government  plans  to  acquire  engineering 
drawings  for  SANR  to  conduct  depot  level 
maintenance  Program  officials  told  us  they  will  also 
ask  for  some  data  rights  for  use  in  building 
government-operated  depots  for  maintaining 
program  software  and  hardware. 

Other  Program  Issues 

Program  officials  stated  that  they  have  developed  a 
revised  acquisition  strategy  and  test  and  evaluation 
master  plan  for  the  program,  which  are  expected  to 
be  approved  mid-201 7,  to  reflect  the  close  out  of  the 
SALT  sub-program  They  added  that  they  have  also 
developed  a  revised  acquisition  program  baseline  to 
reflect  this  change,  which  is  expected  to  be 
approved  in  earty  2018  Program  officials  stated  that 
they  expect  to  complete  an  affordability  and  a 
should-cost  analysis  for  SANR  in  2021  to  support 
the  future  production  contract  award. 


Program  Office  Comments  ^ 

In  commenting  on  a  draft  of  this  assessment,  the  ^ 
program  office  provided  technical  comments,  which 
were  incorporated  where  appropnate 
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©  Program  description 

©  Illustration  or  photo  of  system 

0  Schedule  timeline  identifying  key  dates  for  the  program  including  the 
start  of  development,  major  design  reviews,  production  decisions, 
and  planned  operational  capability 

©  Program  Essentials  Programmatic  information  including  the  prime 
contractor,  program  office  location,  and  funding  needed  to  complete 

@  Program  Performance  Cost  and  schedule  baseline  estimates  and 
the  latest  estimate  provided  as  of  January  2017 


©  Brief  summary  describing  the  program’s  implementation  of 
knowledge-based  acquisition  practices  and  its  current  status 

®  Attainment  of  Product  Knowledge  Depiction  of  selected 
knowledge-based  practices  and  the  program's  progress  in 
attaining  that  knowledge 

<D  Assessment  of  program’s  technology,  design,  and  production 
maturity,  as  well  as  other  program  issues 

O  Program  Office  Comments  General  comments  provided  by  the 
cognizant  program  office 


Source:  GAO  analysis.  |  GAO-1 7-333SP 


For  our  two-page  assessments,  we  depict  the  extent  of  knowledge  gained 
in  a  program  at  the  time  of  our  review  with  a  scorecard  and  narrative 
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summary  at  the  bottom  of  the  first  page  of  each  assessment.  These 
scorecards  display  key  knowledge-based  acquisition  practices  that  should 
be  implemented  by  certain  points  in  the  acquisition  process.  The  more 
knowledge  the  program  has  attained  by  each  of  these  key  points,  the 
more  likely  the  weapon  system  will  be  delivered  within  its  estimated  cost 
and  schedule.  A  knowledge  deficit  means  the  program  is  proceeding 
without  sufficient  knowledge  about  its  technologies,  design,  or 
manufacturing  processes,  and  faces  unresolved  risks  that  could  lead  to 
cost  increases  and  schedule  delays. 

For  each  program,  we  identify  a  knowledge-based  practice  that  has  been 
implemented  with  a  closed  circle.  We  identify  a  knowledge-based  practice 
that  has  not  yet  been  implemented  with  an  open  circle.  If  the  program  did 
not  provide  us  with  enough  information  to  make  a  determination,  we  show 
this  with  a  dashed  line.  A  knowledge-based  practice  that  is  not  applicable 
to  the  program  is  grayed  out.  A  knowledge-based  practice  may  not  be 
applicable  to  a  particular  program  if  the  point  in  the  acquisition  cycle  when 
the  practice  should  be  implemented  has  not  yet  been  reached,  or  if  the 
particular  practice  is  not  relevant  to  the  program.  For  programs  that  have 
not  yet  entered  system  development,  we  show  a  projection  of  knowledge 
attained  for  the  first  three  practices.  For  programs  that  have  entered 
system  development  but  not  yet  held  a  critical  design  review,  we  assess 
actual  knowledge  attained  for  these  three  practices.  For  programs  that 
have  held  a  critical  design  review  but  not  yet  entered  production,  we 
assess  knowledge  attained  for  the  first  five  practices.  For  programs  that 
have  entered  production,  we  assess  knowledge  attained  for  all  eight 
practices. 

We  make  adjustments  to  both  the  key  points  in  the  acquisition  cycle  and 
the  applicable  knowledge-based  practices  for  shipbuilding  programs.  For 
shipbuilding  programs  that  have  not  yet  awarded  a  detail  design  contract, 
we  show  a  projection  of  knowledge  attained  for  the  first  three  practices. 
For  shipbuilding  programs  that  have  awarded  this  contract  but  not  yet 
started  construction,  we  would  assess  actual  knowledge  attained  for 
these  three  practices.  For  shipbuilding  programs  that  have  started 
construction,  we  assess  the  knowledge  attained  for  the  first  four 
practices.  We  do  not  assess  the  remaining  four  practices  for  shipbuilding 
programs  as  they  are  not  applicable  for  these  programs.  See  figure  17  for 
examples  of  the  knowledge  scorecards  we  use  to  assess  these  different 
types  of  programs. 
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Figure  17:  Examples  of  Knowledge  Scorecards 

Program  in  production  Shipbuilding  program 


Attainment  of  Product  Knowledge 


As  of  January  2017 

STATUS  CURRENT 
AT  STATUS 

Resources  and  requirements  match 

•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit.  and  function  within  a  relevant  environment 

DEVELOPMENT 

START 

o 

• 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

o 

• 

•  Complete  a  system-level  preliminary  design  review 

o 

• 

Product  design  is  stable 

•  Release  at  least  90  percent  of  design  drawings 

DESIGN  REVIEW 

o 

• 

•  Test  a  system-level  integrated  prototype 

o 

• 

Manufacturing  processes  are  mature 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

9  or  critical  processes  are  in  statistical  control 

PRODUCTION 

START 

o 

• 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

o 

• 

•  Test  a  production-representative  prototype 

o 

• 

^  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

O  Knowledge  not  attained  Not  applicable 

Attainment  of  Product  Knowledge 


As  of  January  2017 


STATUS  CURRENT 
AT  STATUS 


Resources  and  requirements  match 


DETAIL  DESIGN 
CONTRACT 
AWARD 


•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 


o  • 


•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 


o  o 


•  Complete  a  system-level  preliminary  design  review 

o 

• 

1  Product  design  is  stable 

•  Complete  basic  and  functional  design  to  include 

100  percent  of  3D  product  modeling 

FABRICATION 

START 

o 

• 

^  Knowledge  attained  ■  ■  ■  ■  Information  not  available 
O  Knowledge  not  attained  Not  applicable 


Source:  GAO.  |  GAO-1 7-333SP 


Statement  on  Small 

Business 

Participation 


Pursuant  to  a  mandate  in  a  report  for  the  National  Defense  Authorization 
Act  for  Fiscal  Year  2013,  we  reviewed  whether  individual  subcontracting 
reports  from  a  program’s  prime  contractor  or  contractors  were 
acknowledged — reviewed  by  the  agency  that  awarded  the  contract  and 
found  to  have  no  errors  or  other  issues — on  the  Electronic  Subcontracting 
Reporting  System  (eSRS).36  Federal  law  requires  prime  contractors  to 
make  a  good  faith  effort  to  award  a  portion  of  their  subcontracts  to  small 
businesses  and  in  some  cases  to  have  small  business  subcontracting 
plans.37  We  reviewed  this  information  for  78  of  the  major  defense 


36H.R.  Rep.  No.  112-479,  at  284  (2012). 

37Small  business  subcontracting  plans,  which  are  required  by  the  Small  Business  Act,  15 
U.S.C.  §  637(d),  establish  goals  for  small  business  subcontracting  and  describe  how  the 
contractor  plans  to  achieve  those  goals. 
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acquisition  programs  in  our  assessment  that  reported  contract  information 
in  their  December  2015  Selected  Acquisition  Report  (SAR)  submissions. 
The  contract  numbers  for  each  program’s  prime  contracts  were  entered 
into  the  eSRS  database  to  determine  whether  the  individual 
subcontracting  reports  from  the  prime  contractors  had  been 
acknowledged  by  the  government.  The  government  uses  individual 
subcontracting  reports  on  eSRS  as  one  method  of  monitoring  small 
business  participation,  as  the  report  includes  goals  for  small  business 
subcontracting.  Not  all  prime  contracts  for  major  defense  acquisition 
programs  are  required  to  submit  individual  subcontracting  reports.  For 
example,  some  contractors  report  small  business  participation  at  a 
corporate  level  as  opposed  to  a  program  level  and  these  data  are  not 
captured  in  the  individual  subcontracting  reports.  Information  gathered  for 
this  analysis  is  presented  in  appendix  VI. 
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Army  Programs  Summary 


We  completed  individual  assessments  on  12  of  the  Army’s  20  current  and  future  major  defense  acquisition 
programs.  Of  these  12  programs,  10  are,  for  the  most  part,  in  system  development  or  early  production,  while  2 
are  future  programs  that  DOD  expects  to  enter  system  development  in  the  next  few  years.  We  found  the  Army 
currently  estimates  a  need  of  $64.4  billion  in  funding  to  complete  the  acquisition  of  these  12  programs.  In  addition, 
we  compared  these  1 2  programs’  first  full  estimates  of  cost  and  schedule,  as  available,  with  their  current 
estimates  and  found: 

•  net  cost  growth  totals  $200  million  almost  all  of  which  was  realized  over  the  past  five  years,  and 

•  program  schedule  delays  average  approximately  21  months. 

Further,  2  of  the  12  programs — the  Indirect  Fire  Protection  Capability  Increment  2-Intercept  Block  1  (IFPC  Inc  2-1 
Block  1)  and  Patriot  Advanced  Capability-3  Missile  Segment  Enhancement  (PAC-3  MSE)  programs — completed 
all  the  activities  associated  with  the  applicable  knowledge  based  best  practices  we  assess  as  the  programs 
reach  their  knowledge  points.  Yet,  only  IFPC  Inc  2-1  Block  1  completed  all  applicable  activities  on  time. 


Acquisition  Phase  and  Size  of  the  Currently  Estimated  Acquisition  Cost  Summary  of  Knowledge  Attained  to  Date  for 

12  Programs  Assessed  for  the  12  Programs  Assessed  Programs  Beyond  System  Development  Start 


System  development 


Technology  development 


Fiscal  year  2017  dollars  in  billions 


Programs  in 

technology 

development 


Programs  in 

system 

development 


Cost  and  Schedule  Growth  on  9  Programs 
in  the  Current  Portfolio 

Fiscal  year  2017  dollars  in  billions 


$0.2 


-$1 .7 


Research  and  Procurement  Total 

development  costs  acquisition 

costs  costs 


□  Growth/reduction 
from  first  full  estimate 


[  First  full  estimate 


Program 

common 

name 

Knowledge 
Point  (KP)  1 

Resources 

and 

requirements 

match 

At  Current 
KP1  Status 

Knowledge 
Point  2 

Product 
design 
is  stable 

At  Current 
KP2  Status 

Knowledge 
Point  3 

Manufacturing 
processes  are 
mature 

At  Current 
KP3  Status 

AMF 

o 

• 

• 

■ 

KP  3 
in  future 

AMPV 

o 

• 

o 

o 

KP  3 
in  future 

CIRCM 

o 

o 

• 

• 

KP  3 
in  future 

IAMD 

o 

o 

o 

• 

KP  3 
in  future 

IFPC  Inc  2-1 
Blk  1 

• 

• 

KP  2 
in  future 

KP  3 
in  future 

JAGM 

o 

o 

• 

• 

KP  3 
in  future 

JTRS-HMS 

o 

• 

o 

• 

o  o 

M109A7 

— 

• 

— 

• 

o  o 

PAC-3  MSE 

o 

• 

o 

• 

o  • 

WIN-T  Inc  2 

o 

• 

o  • 

•  All  applicable  knowledge  practices 
completed 

OOne  or  more  applicable  knowledge 
practices  were  not  completed 


Knowledge  practice  is  not  applicable 


Information  not  available  for  knowledge 
practice 


•  Cost  growth  of  more  than  15  percent  and/or 
schedule  delays  of  more  than  6  months 

•  Cost  growth  of  1 5  percent  or  less  and  schedule 
delays  of  6  months  or  less 

No  first  full  estimate  available 


Average 
acquisition 
cycle  time 

(in  months) 


Note:  Bubble  size  is  based  on  each  program’s  currently  estimated 
future  funding  needed. 


Note:  For  acquisition  cycle  time  only  8  programs  were 
assessed  as  not  all  programs  contained  sufficient 
information  within  their  first  full  estimates  to  determine 
acquisition  cycle  time.  In  addition  to  research  and 
development  and  procurement  costs,  total  acquisition 
cost  includes  acquisition  related  operation  and 
maintenance  and  system-specific  military  construction 
costs. 


Source:  GAO  analysis  of  DOD  data.  |  GAO-1 7-333SP 
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Army  Program  Assessments 

2-page  assessments 

Page  Number 

Airborne  and  Maritime/Fixed  Station  (AMF) 

65 

Armored  Multi-Purpose  Vehicle  (AMPV) 

67 

Common  Infrared  Countermeasure  (CIRCM) 

69 

Indirect  Fire  Protection  Capability  Increment  2-Intercept  Block  1  (IFPC  Inc  2-1  Block  1) 

71 

Integrated  Air  and  Missile  Defense  (IAMD) 

73 

Joint  Air-to-Ground  Missile  (JAGM) 

75 

Joint  Tactical  Radio  System  Handheld,  Manpack,  and  Small  Form  Fit  Radios  (JTRS 

HMS) 

77 

Ml  09A7  Family  of  Vehicles  (Ml  09A7  FOV) 

79 

Patriot  Advanced  Capability-3  Missile  Segment  Enhancement  (PAC-3  MSE) 

81 

Warfighter  Information  Network-Tactical  (WIN-T)  Increment  2 

83 

1-page  assessments 

Improved  Turbine  Engine  Program  (ITEP) 

85 

Long  Range  Precision  Fires  (LRPF) 

86 
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Lead  Component:  Army 


Common  Name:  AMF 


Airborne  and  Maritime/Fixed  Station  (AMF) 


The  Army's  AMF  program  plans  to  acquire  non- 
developmental,  software-defined  radios — the  Small 
Airborne  Networking  Radio  (SANR) — and 
associated  equipment  for  integration  into  Army 
rotary  wing  and  unmanned  aerial  systems.  These 
radios  will  provide  simultaneous  voice  and  data 
communications  between  Army  platforms  and 
ground  forces.  The  program  previously  planned  to 
also  acquire  the  Small  Airborne  Link  16  Terminal 
(SALT)  radio,  but,  in  August  201 5,  the  Army  directed 
program  officials  to  close  out  the  SALT  sub¬ 
program.  We  assessed  SANR  only. 


Source:  U.S.  Army. 


System  development  }  Production 


▲ 

▲ 

▲ 

▲ 

▲ 

▲ 

A 

Development 

Design 

GAO 

Low-rate 

Full-rate 

End  operational 

Initial 
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review 

review 
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(3/08) 

(11/09) 

(1/17) 

(4-6/21) 

(1-3/22) 

(7/23) 

(9/23) 

Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  in  millions) 

Prime  contractor:  TBD 

As  of 

Latest 

Percent 

Program  office:  Aberdeen  Proving 

10/2008 

08/2016 

change 

Ground,  MD 

Research  and  development  cost 

$2,109.8 

$1,668.1 

-20.9% 

Funding  needed  to  complete: 

Procurement  cost 

$6,735.0 

$1,645.9 

-75.6% 

R&D:  $71 .8  million 

Total  program  cost 

$8,844.7 

$3,314.1 

-62.5% 

Procurement:  $1 ,645.9  million 

Program  unit  cost 

$.326 

$.233 

-28.6% 

Total  funding:  $1 ,71 7.7  million 

Total  quantities 

27,102 

14,222 

-47.5% 

Procurement  quantity:  14,060 

Acquisition  cycle  time  (months) 

80 

186 

132.5% 

The  quantities  identify  the  total  number  of  channels  required;  currently  one  SANR  radio  is  capable  of 

providing  two  channels. 

In  July  2012,  as  part  of  an  overall  Joint  Tactical 
Radio  System  (JTRS)  reorganization,  DOD 
directed  the  AMF  program  to  pursue  a 
restructured  acquisition  approach  and  acquire  the 
desired  radios  as  a  modified  non-developmental 
item,  leveraging  to  the  maximum  extent  practical 
investments  made  since  2008  within  the  original 
program.  As  a  result,  the  program  will  procure 
existing  radios,  which  will  be  tested  for  technology 
maturity  as  part  of  the  formal  testing  process.  Due 
to  the  program's  non-developmental  strategy,  we 
do  not  have  insight  into  the  percentage  of  design 
drawings  released  and  officials  report  the  testing 
of  a  system-level  integrated  prototype  is  not 
applicable.  The  first  planned  procurement  of 
radios  is  currently  scheduled  for  after  the  non- 
developmental  item  contract  award,  which  is 
expected  in  2019  with  full-rate  production 
decision  scheduled  to  begin  in  2023. 


Attainment  of  Product  Knowledge 


As  of  January  2017 


Resources  and  requirements  match 


STATUS  CURRENT 
AT  STATUS 


•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

DEVELOPMENT 

START 

• 

• 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

o 

• 

•  Complete  a  system-level  preliminary  design  review 

• 

• 

1  Product  design  is  stable 

•  Release  at  least  90  percent  of  design  drawings 

DESIGN  REVIEW 

• 

•  Test  a  system-level  integrated  prototype 

• 

1  Manufacturing  processes  are  mature 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

PRODUCTION 

START 

9  or  critical  processes  are  in  statistical  control 
Demonstrate  critical  processes  on  a  pilot  production  line 
Test  a  production-representative  prototype 


^  Knowledge  attained 
O  Knowledge  not  attained 


Information  not  available 
Not  applicable 
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Common  Name:  AMF 


AMF  Program 

Technology,  Design,  and  Production  Maturity 

In  July  2012 — as  part  of  an  overall  JTRS 
reorganization  of  several  related  programs — the 
Under  Secretary  of  Defense  for  Acquisition, 
Technology  and  Logistics  directed  the  AMF  program 
to  pursue  a  restructured  acquisition  approach  and 
acquire  the  desired  radios  as  a  modified  non- 
developmental  item,  leveraging  to  the  maximum 
extent  practical  prior  investments  made  on  the 
original  program  since  development  started  in  2008. 
Under  the  program's  original  acquisition  strategy, 
AMF  achieved  a  stable  design  by  releasing  at  least 
90  percent  of  its  drawings  and  testing  an  integrated 
prototype  before  it  held  its  critical  design  review. 

The  restructuring  of  the  acquisition  strategy, 
however,  shifted  the  program  from  a  development 
effort  supporting  Army,  Air  Force,  and  Navy 
platforms  to  a  non-developmental  effort  that 
supports  only  Army  aviation  efforts.  Since  the 
government  is  procuring  an  already  existing  item 
from  a  commercial  entity  in  a  non-developmental 
effort,  the  design  knowledge  criteria  related  to 
drawings  and  prototypes  are  no  longer  applicable. 
The  program  officials  have  identified  critical 
technologies  necessary  for  the  existing  radios  the 
Army  intends  to  procure,  and  the  program  plans  to 
have  the  technology  maturity  demonstrated  as  part 
of  the  overall  test  and  demonstration  process.  The 
program  does  not  intend  to  develop  any  new 
technologies  or  software  for  the  radios.  In  2014,  the 
Army  split  AMF  into  two  separate  sub-programs — 
SALT  and  SANR.  In  August  2015,  the  Army  directed 
the  close-out  of  the  SALT  sub-program  as  these 
radios  would  not  have  met  the  Army's  operational 
requirements  until  fiscal  year  2021  and  were 
expected  to  be  more  expensive  than  other  options. 
The  Army  no  longer  intends  to  procure  these  radios. 

The  SANR  sub-program  is  currently  in  the  pre¬ 
solicitation  phase.  No  production  contracts  have 
been  awarded  yet.  Program  officials  stated  that  the 
non-developmental  item  strategy  will  ensure  that  a 
certain  level  of  production  readiness  is  achieved. 
They  added  that  1 1 0  radios,  out  of  the  7,1 1 1 
expected,  will  be  purchased  for  verification  testing 
and  initial  platform  integration.  Reliability  verification 
testing  is  expected  to  begin  after  purchase  of  the 
first  radios  in  fiscal  year  201 9  and  will  be  completed 
prior  to  the  start  of  full-rate  production  in  2023. 


The  government  plans  to  acquire  engineering 
drawings  for  SANR  to  conduct  depot  level 
maintenance.  Program  officials  told  us  they  will  also 
ask  for  some  data  rights  for  use  in  building 
government-operated  depots  for  maintaining 
program  software  and  hardware. 

Other  Program  Issues 

Program  officials  stated  that  they  have  developed  a 
revised  acquisition  strategy  and  test  and  evaluation 
master  plan  for  the  program,  which  are  expected  to 
be  approved  mid-201 7,  to  reflect  the  close  out  of  the 
SALT  sub-program.  They  added  that  they  have  also 
developed  a  revised  acquisition  program  baseline  to 
reflect  this  change,  which  is  expected  to  be 
approved  in  early  2018.  Program  officials  stated  that 
they  expect  to  complete  an  affordability  and  a 
should-cost  analysis  for  SANR  in  2021  to  support 
the  future  production  contract  award. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  technical  comments,  which 
were  incorporated  where  appropriate. 
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Lead  Component:  Army 


Common  Name:  AMPV 


Armored  Multi-Purpose  Vehicle  (AMPV) 

The  Army's  Armored  Multi-Purpose  Vehicle  (AMPV) 
is  the  replacement  to  the  Ml  1 3  family  of  vehicles  at 
the  Brigade  level  and  below.  The  AMPV  will  replace 
the  Ml  13  in  five  mission  roles:  general  purpose, 
medical  evacuation,  medical  treatment,  mortar 
carrier,  and  mission  command.  The  Army 
determined  that  development  of  the  AMPV  is 
necessary  due  to  mobility,  survivability,  and  force 
protection  deficiencies  identified  with  the  Ml  13,  as 
well  as  space,  weight,  power,  and  cooling  limitations 
that  prevent  the  incorporation  of  future 
technologies. 

Source:  U.  S.  Army. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  in  millions) 

Prime  contractor:  BAE  Systems 

As  of 

Latest 

Percent 

Program  office:  Warren,  Ml 

05/2015 

12/2015 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$1,018.0 

$1,043.6 

2.5% 

R&D:  $661 .5  million 

Procurement  cost 

$10,030.5 

$10,030.7 

0.0% 

Procurement:  $10,030.7  million 

Total  program  cost 

$11,048.5 

$11,074.3 

0.2% 

Total  funding:  $10,692.2  million 

Program  unit  cost 

$3,763 

$3,772 

0.2% 

Procurement  quantity:  2,897 

Total  quantities 

2,936 

2,936 

0.0% 

Acquisition  cycle  time  (months) 

87 

87 

0.0% 

The  AMPV  program  held  a  critical  design  review 
in  June  2016,  with  its  critical  technologies  mature 
and  a  design  that  was  stable,  but  not 
demonstrated.  Following  this  review,  the  program 
received  approval  to  modify  a  key  performance 
requirement  and  several  system  attributes,  which 
DOD  had  previously  judged  AMPV  would  be 
challenged  to  meet.  To  support  design 
demonstration,  BAE  Systems  started  fabricating 
prototype  vehicles  in  May  2016,  with  the  first 
being  delivered  in  December  2016.  Delays 
related  to  engineering  drawing  releases, 
manufacturing  planning,  and  ballistic  weld 
certification  have  impeded  fabrication  work, 
although  program  officials  noted  these  issues 
have  largely  been  overcome.  The  program  has 
also  identified  and  is  mitigating  interface  issues 
between  AMPV  and  certain  platforms  it  will  rely 
upon  for  communication  and  networking 
capabilities. 
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Demonstrate  Manufacturing  Readiness  Level  of  at  least 
9  or  critical  processes  are  in  statistical  control 

Demonstrate  critical  processes  on  a  pilot  production  line 
Test  a  production-representative  prototype 


PRODUCTION 

START 


^  Knowledge  attained 
o  Knowledge  not  attained 


Information  not  available 
Not  applicable 


Common  Name:  AMPV 


AMPV  Program 

Technology  and  Design  Maturity 

The  AMPV  program  entered  system  development  in 
December  2014  with  its  critical  technologies  fully 
mature  as  determined  by  an  independent  review 
team.  According  to  program  officials,  while  the 
AMPV  design  utilizes  a  new  hull  design,  a  majority 
of  subsystems  are  derived  from  existing  vehicle 
designs. 

Although  the  program  held  its  critical  design  review 
in  June  2016  with  over  90  percent  of  its  design 
drawings  released  to  manufacturing,  the  program 
had  not  yet  demonstrated  a  system-level  integrated 
prototype.  BAE  Systems  started  fabricating  AMPV 
prototype  vehicles  in  May  2016  to  support  design 
demonstration  and  developmental  testing.  However, 
delays  related  to  releasing  engineering  drawings 
and  manufacturing  planning  efforts  disrupted  these 
fabrication  activities.  Program  officials  report  that 
these  issues  are  being  addressed  and  the  program 
delivered  its  first  prototype  in  December  2016. 

Prior  to  AMPV’s  critical  design  review,  the  Office  of 
the  Deputy  Assistant  Secretary  of  Defense  for 
Systems  Engineering  assessed  that  the  AMPV 
preliminary  design  would  be  challenged  to  meet 
survivability  and  force  protection  requirements  and 
identified  a  need  to  modify  these  requirements  to 
match  the  capabilities  that  the  AMPV  was  likely  to 
provide.  In  response,  the  program  requested  and 
received  approval  from  the  Army  Requirements 
Oversight  Council  to  modify  the  system’s 
survivability  requirement  as  well  as  several  key 
system  attributes.  In  September  2016,  the  Joint 
Requirements  Oversight  Council  validated  the 
survivability  requirement  change.  AMPV’s 
survivability  and  force  protection  is  still  expected  to 
exceed  that  of  the  Ml  13  vehicle  it  is  intended  to 
replace. 

Production  Maturity 

Although  the  AMPV  program  expects  to 
demonstrate  its  critical  manufacturing  processes  on 
a  pilot  production  line,  it  does  not  anticipate  that 
these  processes  will  achieve  statistical  control  prior 
to  production  start  in  February  2019.  In  addition, 
BAE  Systems  has  had  difficulty  attaining  ballistic 
weld  certification — essential  for  the  government 
ensuring  that  the  contractor  can  successfully  weld 
ballistic  materials — due  to  qualification  test  failures. 


While  this  delayed  deliveries  of  ballistic  hulls,  a  key 
vehicle  component,  this  issue  has  been  resolved 
according  to  program  officials.  BAE  redesigned 
these  welds  and  validated  their  design  in  August 
2016. 

Other  Program  Issues 

AMPV  is  dependent  on  other  programs  for  its  key 
communication  and  networking  capabilities.  While 
most  systems  providing  these  capabilities  exist  on 
current  platforms,  the  program  has  unresolved 
interface  issues  with  some  of  them.  For  example, 
the  Manpack  radios — separately  produced  as  part 
of  the  Army's  Joint  Tactical  Radio  System 
Handheld,  Manpack,  and  Small  Form  Fit  (JTRS 
HMS)  Radios  program — requires  design  changes  to 
be  fully  compatible  with  AMPV's  interface.  To 
mitigate  the  Manpack  radio  integration  risk,  program 
officials  plan  to  include  a  legacy  radio  platform  in  the 
AMPV's  design.  However,  if  other  communications 
and  networking  systems — some  of  which  have 
experienced  their  own  developmental  challenges — 
are  not  available  when  needed  to  support  AMPV's 
testing  and  fielding,  the  Army  will  likely  face  a 
choice  of  adjusting  the  AMPV's  schedule  or 
accepting  reduced  capabilities. 

Program  Office  Comments 

In  addition  to  providing  technical  comments,  which 
we  incorporated  where  appropriate,  the  program 
office  noted  that  the  program  successfully  rolled-out 
the  first  AMPV  prototype  on-time  in  December 

2016.  Further,  the  program  office  stated  that  the 
prototypes  are  built  on  existing  production  lines 
using  production-representative  work  instructions, 
material,  manpower,  and  tooling.  These 
manufacturing  processes  are  being  continuously 
updated  and  refined  as  prototypes  move  down  the 
line,  and  will  stabilize  and  mature  in  fiscal  year 

2017.  The  program  successfully  completed  ballistic 
weld  joint  redesign  and  testing  to  confirm  the  armor 
weldments  met  ballistic  shock  requirements.  The 
associated  welding  procedures  used  for  building 
hulls  are  now  fully  certified  and  there  are  no 
schedule  delays.  The  program  is  coordinating  with 
other  program  offices  to  understand  the  detailed 
interfaces  required  to  ensure  seamless  integration 
of  emerging  radios  and  does  not  currently  anticipate 
any  program  schedule  delays. 
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Lead  Component:  Army 


Common  Name:  CIRCM 


Common  Infrared  Countermeasure  (CIRCM) 

The  Army's  CIRCM  is  the  next  generation  of  - 

Advanced  Threat  Infrared  Countermeasures 


(ATIRCM)  designed  to  defend  aircraft  from  infrared- 
guided  missiles.  The  program  is  developing  a  laser- 
based  system  for  use  with  a  missile  warning  system 
and  countermeasure  dispenser  that  deploys 
expendables,  such  as  flares  and  chaff.  CIRCM  will 
be  installed  on  rotary-wing,  tilt-rotor,  and  small  fixed- 
wing  aircraft  across  DOD.  CIRCM  was  originally 
started  as  a  subprogram  under  the 
ATIRCM/Common  Missile  Warning  System. 


Source:  Northrop  Grumman. 
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Program  Essentials 


Program  Performance  (fiscal  year  2017  dollars  in  millions) 


Prime  contractor:  Northrop  Grumman 
Program  office:  Huntsville,  Alabama 
Funding  needed  to  complete: 

R&D:  $379.7  million 
Procurement:  $1 ,836.3  million 
Total  funding:  $2,216.0  million 
Procurement  quantity:  1 ,076 


Research  and  development  cost 
Procurement  cost 
Total  program  cost 
Program  unit  cost 
Total  quantities 

Acquisition  cycle  time  (months) 


As  of 

Latest 

Percent 

07/2016 

10/2016 

change 

$777.4 

$777.4 

0.0% 

$1,836.3 

$1,836.3 

0.0% 

$2,613.7 

$2,613.7 

0.0% 

$2,325 

$2,325 

0.0% 

1,124 

1,124 

0.0% 

115 

115 

0.0% 

The  CIRCM  program  entered  system 
development  in  August  2015  with  its  critical 
technologies  approaching  maturity  based  on  the 
results  of  a  technology  readiness  assessment 
conducted  in  December  2014.  The  independent 
review  team  was  not  able  to  determine  whether 
any  of  the  critical  technologies  achieved  full 
maturity  as  testing  of  the  CIRCM  prototypes  in  an 
operational  environment  was  not  performed.  The 
program  completed  its  critical  design  review  in 
October  201 6,  with  a  stable  design  and  almost  all 
drawings  released  to  manufacturing.  The 
program  office  has  identified  key  manufacturing 
risk  areas  and  included  them  in  its  plans  for  a  pilot 
line  validation  and  production  readiness  review  to 
assess  the  program’s  readiness  to  enter 
production. 
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Attainment  of  Product  Knowledge 


As  of  January  2017 

STATUS  CURRENT 
AT  STATUS 

Resources  and  requirements  match 

DEVELOPMENT 

•  Demonstrate  all  critical  technologies  are  very  close  to 

START 

A  A 

final  form,  fit,  and  function  within  a  relevant  environment 

•  • 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 

o  o 

function  within  a  realistic  environment 

•  Complete  a  system-level  preliminary  design  review 

•  • 

_ 

Product  design  is  stable 

DESIGN  REVIEW 

•  Release  at  least  90  percent  of  design  drawings 

•  • 

•  Test  a  system-level  integrated  prototype 

•  • 

Manufacturing  processes  are  mature 

PRODUCTION 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

START 

9  or  critical  processes  are  in  statistical  control 

■  ■ 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 

£  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

O  Knowledge  not  attained  Not  applicable 

GAO-1 7-333SP  Assessments  of  Selected  Weapon  Programs 


Common  Name:  CIRCM 


CIRCM  Program 

Technology  Maturity 

The  CIRCM  program  entered  system  development 
in  August  2015  with  its  critical  technologies 
approaching  maturity.  In  December  2014,  an 
independent  review  team  conducted  an  assessment 
of  the  program's  nine  critical  technologies  in  a 
relevant  environment — including  the  gimbal 
assembly,  camera  assembly,  the  quantum  cascade 
laser,  and  others — and  determined  that  all 
technologies  were  nearing  maturity.  However, 
according  to  a  November  2016  program  document, 
none  of  the  technologies  have  achieved  maturity 
yet.  The  program  originally  planned  to  enter  the 
acquisition  cycle  at  system  development.  However, 
after  competitive  prototyping,  the  program 
determined  none  of  the  vendors  had  built  prototypes 
with  the  required  technology  readiness  levels  and 
all  required  major  redesigns  to  meet  weight 
requirements.  The  key  risks  going  into  the 
technology  development  phase  were  the  CIRCM's 
weight,  probability  of  countermeasure,  and 
reliability.  The  program  entered  the  acquisition  cycle 
at  technology  development,  which  resulted  in 
allowing  vendors  3  years  to  further  develop  their 
designs.  Government  test  results  indicate  that 
weight,  probability  of  countermeasure,  and  reliability 
are  meeting  or  exceeding  exit  criteria  and/or  the 
system  requirements. 

Design  Maturity 

The  program  completed  its  critical  design  review 
(CDR)  in  October  2016  and  reports  that  the  design 
is  stable  based  on  the  number  of  design  drawings 
released  to  manufacturing.  One  hundred  percent  of 
the  drawings  for  the  CIRCM  system  and  95  percent 
of  the  drawings  for  the  modification  kits — which 
include  the  hardware,  wiring  harness,  and  cables 
necessary  to  install  CIRCM  on  each  aircraft — have 
been  delivered.  The  program  office  had  one  critical 
action  from  the  CDR  and  closed  it  in  January  2017. 

Production  Maturity 

According  to  program  officials,  key  manufacturing 
risk  areas  have  been  identified  and  included  in  the 
program's  pilot  line  validation  and  production 
readiness  review  plans.  Both  of  these  events  will 
cover  all  manufacturing  concerns  to  determine  the 
program's  readiness  to  enter  production.  Pilot  line 
validations  are  conducted  to  validate  manufacturing 
capabilities  are  in  place  to  produce  CIRCM  units 


and  address  manufacturing  maturity  progression 
toward  achieving  a  production  capability  and 
capacity  that  will  meet  full  rate  production 
requirements.  This  validation  will  be  conducted 
while  initial  CIRCM  units  are  manufactured  so  that 
instructions,  equipment,  tooling,  test  equipment, 
and  workers  can  be  observed.  The  Production 
Readiness  Review,  scheduled  for  August  2017, 
determines  if  the  system  design  is  ready  for 
production  and  if  the  developer  has  accomplished 
adequate  production  planning  for  entering  both  low- 
rate  initial  production  and  full-rate  production. 

Program  Office  Comments 

The  program  office  reports  that  it  completed  CDR  in 
October  2016  and  is  preparing  for  pilot  line 
validations  and  production  readiness  reviews  to 
validate  the  contractor's  ability  to  manufacture  the 
final  CIRCM  design  in  the  quantities  required.  The 
CIRCM  program  has  begun  building  production 
representative  articles  based  on  the  design 
presented  at  the  design  review,  which  will  begin 
system  level  developmental  testing  in  preparation 
for  operational  testing  scheduled  for  fiscal  year 
2019.  According  to  program  officials,  CIRCM 
remains  on  track  for  completion  of  the  system 
development  phase  and  movement  towards  full-rate 
production  readiness  within  the  cost,  schedule,  and 
performance  requirements  set  forth  in  its  Acquisition 
Program  Baseline.  The  program  also  provided 
technical  comments,  which  were  incorporated 
where  deemed  appropriate. 
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Lead  Component:  Army 


Common  Name:  IFPC  Inc  2-1  Block  1 


Indirect  Fire  Protection  Capability  Increment  2-Intercept  Block  1  (IFPC  Inc  2-1  Block  1)) 

The  Army's  IFPC  Inc  2-1  is  a  follow-on  effort  to 
enhance  and  extend  the  range  of  the  first  IFPC 
increment,  which  provided  a  short-range  capability 
to  counter  threats  from  rockets,  artillery,  and 
mortars.  IFPC  Inc  2-1  consists  of  four  separate  sub¬ 
systems:  an  existing  sensor;  an  interceptor;  a 
command  and  control  system;  and  a  new  multi¬ 
mission  launcher  (MML)  being  developed  by  the 
Army.  IFPC  Inc  2-1  consists  of  three  blocks.  Block  1 
adds  the  capability  to  counter  cruise  missiles  and 
unmanned  aircraft.  We  assessed  Inc  2-1  Block  1. 


Source:  U.S.  Army. 
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Program  Essentials 

Prime  contractors:  Aviation  and  Missile 
Research,  Development  and 
Engineering  Center;  Letterkenny  Army 
Depot 

Program  office:  Huntsville,  Alabama 
Funding  needed  to  complete: 

R&D:  $404.3  million 
Procurement:  $2,729.6  million 
Total  funding:  $3,133.9  million 
Procurement  quantity:  368 


Program  Performance  (fiscal  year  2017  dollars  in  millions) 


As  of 

Latest 

09/2016 

Percent 

change 

Research  and  development  cost 

NA 

$736.8 

NA 

Procurement  cost 

NA 

$2,729.6 

NA 

Total  program  cost 

NA 

$3,466.5 

NA 

Program  unit  cost 

NA 

$9,420 

NA 

Total  quantities 

NA 

368 

NA 

Acquisition  cycle  time  (months) 

NA 

46 

NA 

The  IFPC  Inc  2-1  Block  1  program  plans  to  enter 
development  with  no  critical  technologies.  The 
program  has  released  all  design  drawings  and 
considers  its  system  design  stable  ahead  of  the 
planned  May  2017  critical  design  review. 
Plowever,  this  stability  may  be  compromised  if 
changes  are  needed  before  the  May  2017  critical 
design  review  to  integrate  the  MML  with  the  other 
three  subsystems.  The  program  continues  to 
develop  the  MML  at  two  Army  facilities,  a  strategy 
that  was  previously  determined  to  be  the  most 
cost  effective.  Program  officials  acknowledged 
that  they  have  a  limited  amount  of  time  and 
flexibility  in  their  current  development  schedule  to 
correct  any  deficiencies  identified  during 
developmental  testing.  As  a  result,  this  optimistic 
schedule  could  leave  the  program  at  risk  for 
unplanned  cost  increases  and  schedule  delays  to 
correct  any  such  deficiencies  that  emerge. 


Attainment  of  Product  Knowledge 

...  STATUS  CURRENT 

As  of  January  201 7  at  status 

I  Resources  and  requirements  match 

DEVELOPMENT 

•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

•  Complete  a  system-level  preliminary  design  review 

• 

• 

I  Product  design  is  stable 

DESIGN  REVIEW 

•  Release  at  least  90  percent  of  design  drawings 

•  Test  a  system-level  integrated  prototype 

I  Manufacturing  processes  are  mature 

PRODUCTION 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

9  or  critical  processes  are  in  statistical  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 

^  Knowledge  attained  ■  ■  ■  ■  Information  not  available 
O  Knowledge  not  attained  Not  applicable 
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Common  Name:  IFPC  Inc  2-1  Block  1 


IFPC  Inc  2-1  Block  1  Program 

Technology  Maturity 

In  May  2016,  the  Army  completed  a  technology 
readiness  assessment  for  the  IFPC  Inc  2-1  Block  1 
program  that  identified  no  critical  technologies 
ahead  of  the  program’s  planned  entry  into  system 
development.  Although  three  of  the  four  sub¬ 
systems  that  comprise  IFPC  Inc  2-1  Block  1  are 
either  fielded  or  in  production,  officials 
acknowledged  that  these  systems  pose  potential 
integration  challenges  to  Block  1  system 
development  because  their  capabilities  will  be  used 
in  new  ways  to  prosecute  different  threats.  The  AIM- 
9X  missile,  the  designated  interceptor,  was 
originally  developed  as  an  air-to-air  missile  and 
designed  to  be  fired  from  Navy  tactical  aircraft.  The 
Army  is  relying  on  its  Integrated  Air  and  Missile 
Defense  program  to  produce  the  command  and 
control  system,  which  is  currently  designed  to 
control  the  Patriot  missile  launcher  and  radar 
system.  Previously,  we  found  the  IAMD  program 
faces  its  own  development  challenges.  The 
program  office  also  intends  to  use  the  existing 
Sentinel  air  defense  radar  as  its  sensor  component. 

The  Army  will  develop  and  produce  the  fourth 
system — the  multi-mission  launcher  (MML) — at  two 
facilities:  the  Aviation  and  Missile  Research, 
Development,  and  Engineering  Center  (AMRDEC) 
and  Letterkenny  Army  Depot  (LEAD).  According  to 
program  officials,  AMRDEC  is  responsible  for 
developing  the  MML  units  used  for  testing  during 
system  development,  whereas  LEAD  will  assume 
responsibility  for  MML  low-rate  initial  production. 

Design  Maturity 

According  to  program  officials,  the  program  has 
released  all  of  its  expected  engineering  drawings. 
However,  program  officials  stated  the  prototype 
design  of  the  MML  presented  at  the  program's 
preliminary  design  review  in  September  2015  is 
being  assessed  and  updated,  as  necessary,  in 
preparation  for  the  critical  design  review  (CDR)  in 
May  2017.  The  CDR  will  evaluate  the  integration  of 
the  MML  with  the  existing  sensor,  interceptor,  and 
command  and  control  system.  The  program  has  no 
plans  for  additional  prototyping  during  system 
development  following  its  CDR. 


Program  officials  acknowledged  the  program 
schedule  is  oriented  on  successful  testing  outcomes 
throughout  system  development.  As  a  result, 
program  officials  acknowledge  there  is  limited  time 
available  to  correct  any  deficiencies  that  may  be 
found  during  testing  before  production  begins.  This 
optimistic  schedule  could  leave  the  program  at  risk 
for  unplanned  cost  increases  and  schedule  delays 
to  correct  those  deficiencies. 

Production  Maturity 

Program  officials  stated  that  LEAD  has  the 
capability  and  capacity  to  manufacture  60  MML 
units  per  year,  but  they  are  unsure  how  easily  the 
depot  can  exceed  that  number.  The  current 
production  schedule  increases  the  annual  MML 
production  rate  to  a  maximum  of  72  units  in  fiscal 
year  2022.  Program  officials  stated  they  are 
committed  to  utilizing  Army  facilities  for  MML 
development  and  production  and  are  developing 
mitigation  strategies,  including  identifying  additional 
Army  sources  for  production  tooling  equipment  to 
supplement  the  equipment  already  available  at 
LEAD.  However,  program  officials  stated  that, 
should  they  determine  LEAD  is  unable  to 
manufacture  the  MML  at  required  rates,  they  have 
the  option  of  soliciting  proposals  from  contractors 
for  MML  production  ahead  of  the  IFPC  Inc  2-1  Block 
1  full  rate  production  decision  in  2020. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  technical  commments, 
which  were  incorporated  where  deemed 
appropriate. 
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Lead  Component:  Army 


Common  Name:  IAMD 


Integrated  Air  and  Missile  Defense  (IAMD) 

The  Army's  Integrated  Air  and  Missile  Defense 
(IAMD)  program  is  being  developed  to  network 
sensors,  weapons,  and  a  common  battle  command 
system  across  an  integrated  fire  control  network  to 
support  the  engagement  of  air  and  missile  threats. 

The  IAMD  battle  command  system  will  provide  a 
capability  to  control  and  manage  IAMD  sensors  and 
weapons,  such  as  the  Sentinel  radar  and  Patriot 
launcher  and  radar,  through  an  interface  module 
that  supplies  battle  management  data  and  enables 
networked  operations. 


Source:  Northrop  Grumman. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  In  millions) 

Prime  contractors:  Northrop  Grumman 

As  of 

Latest 

Percent 

Space  &  Mission  Systems  Corp., 

12/2009 

08/2016 

change 

Raytheon  Company 

Research  and  development  cost 

$1,730.2 

$2,678.6 

54.8% 

Program  office:  Redstone  Arsenal,  AL 

Procurement  cost 

$3,724.2 

$3,822.7 

2.6% 

Funding  needed  to  complete: 

Total  program  cost 

$5,454.4 

$6,501.2 

19.2% 

R&D:  $715.7  million 

Program  unit  cost 

$18,427 

$14,675 

-20.4% 

Procurement:  $3,802.0  million 

Total  quantities 

296 

443 

49.7% 

Total  funding:  $4,517.7  million 

Acquisition  cycle  time  (months) 

80 

105 

31.3% 

Procurement  quantity:  427 

IAMD  critical  technologies  achieved  full  maturity 
in  2016 — more  than  6  years  after  development 
start.  Best  practices  recommend  that  technology 
maturity  be  achieved  prior  to  starting 
development.  The  program's  low-rate  production 
decision  has  been  delayed  more  than  4  months  to 
allot  more  time  for  developmental  testing. 
According  to  program  officials,  the  root  causes  for 
this  delay  include  the  program  office's  inability  to 
schedule  test  range  time  as  well  as  the  extra  time 
required  for  review  of  recent  testing  results. 
Program  officials  noted  that  it  will  be  challenging 
to  start  operational  testing  by  the  planned 
September  2017  date  and  anticipate  changes  to 
developmental  and  operational  testing  schedules 
going  forward. 
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Attainment  of  Product  Knowledge 


As  of  January  2017 


Resources  and  requirements  match 


Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 


STATUS  CURRENT 
AT  STATUS 


DEVELOPMENT 

START 


O  O 


Complete  a  system-level  preliminary  design  review 


Product  design  is  stable 


Release  at  least  90  percent  of  design  drawings 
Test  a  system-level  integrated  prototype 


DESIGN  REVIEW 


o 

o 


Manufacturing  processes  are  mature 


Demonstrate  Manufacturing  Readiness  Level  of  at  least 
9  or  critical  processes  are  in  statistical  control 

Demonstrate  critical  processes  on  a  pilot  production  line 
Test  a  production-representative  prototype 


PRODUCTION 

START 


^  Knowledge  attained 
o  Knowledge  not  attained 


Information  not  available 
Not  applicable 


Common  Name:  IAMD 


IAMD  Program 

Technology  Maturity 

As  of  September  2016,  the  IAMD  program 
demonstrated  that  all  of  its  critical  technologies — 
integrated  battle  command,  integrated  defense 
design,  integrated  fire  control  network,  and 
distributed  track  management — are  fully  mature. 
This  achievement  comes  over  6  years  after  the  start 
of  system  development — a  schedule  that  is 
inconsistent  with  best  practices. 

Design  Maturity 

IAMD  completed  its  critical  design  review  with  a 
stable  design  in  2012,  but  the  total  number  of 
design  drawings  has  increased,  with  program 
officials  reporting  that  5  percent  of  them  still  need  to 
be  released  before  production  starts.  Program 
officials  report  they  have  released  95  percent  of 
lAMD's  design  drawings.  In  addition,  IAMD  software 
development  delays,  which  we  have  previously 
reported,  persist.  Program  officials  stated  that,  most 
recently,  the  program  experienced  software 
challenges  going  into  an  IAMD  operational 
assessment  with  some  software  upgrades 
remaining  undelivered.  However,  these  officials 
noted  the  IAMD  program  will  need  software  updates 
on  a  continuing  basis  due  to  threat  changes.  The 
program  is  also  currently  developing  a  software 
version  (version  4.0)  that  officials  anticipate  will 
incorporate  fixes  for  previous  software  deficiencies. 

Production  Maturity 

The  Army  has  delayed  the  IAMD  low-rate 
production  decision — previously  planned  for  August 
2016 — to  allot  more  time  for  developmental  testing. 
According  to  program  officials,  the  root  causes  for 
this  delay  include  limited  access  to  needed  test 
resources,  such  as  test  ranges,  as  well  as  extra 
time  required  for  reviewing  the  limited  user  test 
results.  IAMD  full-rate  production  start  remains 
planned  for  fiscal  year  2018. 

Other  Program  Issues 

The  program  conducted  "limited  user  testing"  from 
March  to  May  2016  to  collect  data  on  the  lAMD's 
system  of  systems  operational  effectiveness, 
suitability,  and  survivability.  This  assessment  of 
IAMD  was  designed  to  evaluate  its  ability  to  defeat 
aerial  threats  in  an  operationally  realistic 
environment  and  determine  hardware  and  software 
maturity,  among  other  goals. 


As  a  result  of  the  limited  user  testing,  the  Army 
Operational  Test  Command  made  several 
recommendations  to  address  issues  with  lAMD's 
operational  effectiveness,  suitability,  and 
survivability.  These  recommendations  are  based  on 
tester  observations  made  during  the  test  and  do  not 
reflect  the  Army  Test  and  Evaluation  Command's 
final  evaluation  recommendations,  which  the  U.S. 
Army  Evaluation  Center  is  currently  developing 
based  on  the  results  of  this  test.  According  to 
program  officials,  IAMD  completed  successful  flight 
tests  in  2016,  which  demonstrated  new  capabilities. 
For  example,  officials  noted  a  successful  first  flight 
test  where  a  PAC-3  missile  intercepted  a  tactical 
ballistic  missile  using  IAMD  software  and  a 
composite  tracker. 

Program  officials  noted  it  will  be  challenging  to  start 
operational  testing  by  September  2017.  They  stated 
that  lAMD's  developmental  and  operational  testing 
schedules  are  likely  to  change.  The  program's  initial 
operational  capability  date  has  been  moved  from 
June  201 8  to  September  2018,  but  program  officials 
stated  that  this  is  still  within  the  program's 
acquisition  program  baseline  parameters. 

According  to  program  officials,  this  minor  schedule 
change  will  provide  the  program  more  time  to  test, 
analyze,  fix,  and  re-test. 

Program  Office  Comments 

The  limited  user  testing  provided  less  than 
satisfactory  results  per  the  Army  Test  and 
Evaluation  Command's  final  assessment.  As  a 
result,  the  Army  Acquisition  Executive  placed  the 
Milestone  C  decision  on  hold  until  the  software 
deficiencies  identified  were  resolved.  According  to 
program  officials,  the  Milestone  C  decision  is 
expected  to  occur  in  2017.  Officials  also  noted  the 
initial  operational  capability  date  will  be  adjusted 
accordingly  to  accommodate  the  necessary  initial 
operational  testing  needed  prior  to  fielding  of  the 
system. 
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Lead  Component:  Army 


Common  Name:  JAGM 


Joint  Air-to-Ground  Missile  (JAGM) 

The  Joint  Air-to-Ground  Missile  is  an  Army-led 
program  with  joint  requirements  from  the  Navy  and 
Marine  Corps.  The  missile  is  designed  to  be  air- 
launched  from  helicopters  and  unmanned  aircraft 
systems  to  target  tanks,  light  armored  vehicles, 
missile  launchers,  bunkers,  and  buildings.  It  is 
intended  to  provide  precision  attack  capabilities  no 
matter  the  time  of  day  or  weather  conditions.  JAGM 
will  replace  all  Hellfire  missile  variants. 


Source:  U.S.  Army. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  in  millions) 

Prime  contractor:  Lockheed  Martin 

As  of 

Latest 

Percent 

Program  office:  Redstone  Arsenal,  AL 

09/2015 

08/2016 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$1,008.0 

$996.3 

-1.2% 

R&D:  $95.5  million 

Procurement  cost 

$4,833.0 

$4,858.0 

0.5% 

Procurement:  $4,831 .9  million 

Total  program  cost 

$5,841.0 

$5,854.3 

0.2% 

Total  funding:  $4,927.4  million 

Program  unit  cost 

$.221 

$.221 

0.2% 

Procurement  quantity:  26,319 

Total  quantities 

26,437 

26,437 

0.0% 

Acquisition  cycle  time  (months) 

38 

38 

0.0% 

The  Army  has  fully  matured  all  three  of  the  JAGM 
program's  critical  technologies.  Following  a  2012 
program  restructuring,  the  Army  redesigned 
JAGM  to  employ  existing  Hellfire  missile 
components.  DOD  subsequently  waived  the 
requirement  to  complete  a  preliminary  design 
review  (PDR)  of  the  redesigned  missile  prior  to 
development  start.  The  Army  held  a  critical  design 
review  (CDR)  in  January  2016,  which  assessed 
the  JAGM  design  as  stable.  The  program  has 
delayed  initial  operational  testing  due  to  a 
requirement  for  new  Apache  platform  software 
and  will  instead  begin  a  limited  user  test  in  May 
2017.  The  program  plans  to  manufacture  JAGM 
on  the  existing  production  line  used  for  Hellfire 
missiles,  and  program  officials  told  us  they  expect 
all  production  processes  to  be  mature  prior  to  the 
start  of  low-rate  production  in  July  2017. 
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Attainment  of  Product  Knowledge 


As  of  January  2017 


Resources  and  requirements  match 


Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 


STATUS  CURRENT 
AT  STATUS 


DEVELOPMENT 

START 


O  • 

O  O 


Complete  a  system-level  preliminary  design  review 


Product  design  is  stable 


Release  at  least  90  percent  of  design  drawings 
Test  a  system-level  integrated  prototype 


DESIGN  REVIEW 


Manufacturing  processes  are  mature 


Demonstrate  Manufacturing  Readiness  Level  of  at  least 
9  or  critical  processes  are  in  statistical  control 

Demonstrate  critical  processes  on  a  pilot  production  line 
Test  a  production-representative  prototype 


PRODUCTION 

START 


^  Knowledge  attained 
o  Knowledge  not  attained 


Information  not  available 
Not  applicable 


Common  Name:  JAGM 


JAGM  Program 

Technology  Maturity 

JAGM  has  three  critical  technologies — the  guidance 
seeker  assembly/sensor  platform,  sensor  software, 
and  mission  software,  all  of  which  the  program 
assessed  as  mature  following  two  successful  test 
shot  events  in  June  2016.  JAGM  components, 
including  the  motor,  warhead,  and  electronics,  are 
common  with  the  existing  Hellfire  missile.  The 
Army's  decision  to  include  these  components  in  the 
JAGM  design  followed  a  2012  restructuring  of  the 
program,  which  included  extending  technology 
development  by  more  than  2  years  to  address 
affordability  concerns  and  risk  reduction  needs. 

Design  Maturity 

The  Army  completed  a  system-level  PDR  for  JAGM 
in  June  2010.  Following  the  program's  2012 
restructuring,  the  Army  implemented  design 
changes  to  the  missile,  but  DOD  waived  the 
requirement  to  conduct  a  PDR  for  the  new  design 
before  the  start  of  system  development.  At  the  CDR 
in  January  2016,  the  program  reported  that  the 
number  of  drawings  expected  had  increased  from 
1 80  to  1 98.  Program  officials  reported  that  nearly  97 
percent  of  these  drawings  have  been  released  and 
that  the  increase  in  drawings  was  due  to  design 
changes  to  existing  hardware  as  well  as  a  new 
drawing  for  the  guidance  section.  The  CDR 
determined  that  the  design  was  stable  and  able  to 
meet  system  performance  requirements.  The 
program  held  five  flight  tests  of  a  system  level 
prototype  prior  to  the  CDR  and  held  an  additional 
two  tests  in  2016,  including  the  first  missile  test  from 
a  MQ-1C  Gray  Eagle. 

JAGM  officials  reported  a  5  month  delay  to  the 
production  of  engineering  and  manufacturing 
development  missiles  because  of  a  manufacturing 
defect  in  the  seeker  head  assembly's  dual-sensor 
interface  circuit  card,  which  may  have  resulted  in 
the  failure  of  the  sensor.  The  contractor  has  since 
redesigned  the  interface  to  new  specifications.  The 
program  also  found  that  the  material  used  to  coat  a 
portion  of  the  transceiver  failed  under  vibration 
loads.  The  contractor  plans  to  use  a  different 
coating  beginning  with  the  81st  testing  missile  it 
produces,  but  program  officials  do  not  anticipate 
performance  limitations  to  earlier  missiles  or 
schedule  disruptions. 


Production  Maturity 

JAGM  will  be  manufactured  at  the  same  facility  as 
the  Hellfire  missile.  The  program  office  is  tracking 
15  different  metrics  related  to  hardware  and 
software  to  assess  JAG  M's  readiness  for 
production.  According  to  program  officials,  these 
metrics  currently  meet  expected  values,  and  the 
program  expects  all  production  processes  to  be 
mature  prior  to  the  start  of  low-rate  production  in 
July  2017. 

Other  Program  Issues 

The  Army  has  delayed  initial  operational  testing  and 
evaluation  (IOT&E)  by  2  years  because  the  AH-64 
Apache  helicopter's  current  software — used  to 
launch  Hellfire  missiles — requires  more  pilot  input  to 
access  JAGM  functionality  than  expected.  New 
platform  software  in  development  for  integration  and 
testing  in  fiscal  year  2018  will  enable  pilots  to  select 
the  full  range  of  options  with  far  less  physical 
interface  with  the  platform.  Flight  testing  of  this 
software  is  planned  for  August  2018  to  be  followed 
by  IOT&E.  In  the  interim,  the  program  has  planned  a 
limited  user  test  in  May  201 7  of  1 0  missile  shots 
with  existing  platform  software  and  hardware  to 
demonstrate  the  missile's  capabilities.  Program 
officials  told  us  they  do  not  expect  any  material 
changes  to  the  missile  between  the  limited  user  test 
and  IOT&E. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  technical  comments,  which 
were  incorporated  where  appropriate. 
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Lead  Component:  Army 


Common  Name:  JTRS  HMS 


Joint  Tactical  Radio  System  Handheld,  Manpack,  and  Small  Form  Fit  Radios  (JTRS  HMS) 

DOD's  Joint  Tactical  Radio  System  (JTRS) 
program,  now  restructured,  was  developing 
software-defined  radios  to  interoperate  with  existing 
radios  and  increase  communications  and 
networking  capabilities.  The  Army's  JTRS  HMS 
program  continues  efforts  to  procure  two  radios — 
the  Rifleman  and  Manpack  radios.  A  subset  of 
Manpack  radios  are  designed  to  operate  with  the 
Mobile  User  Objective  System  (MUOS) — a  Navy 
satellite  communication  system  planned  to  serve  a 
worldwide,  multiservice  population  of  mobile  and 
fixed-site  terminal  users. 


Source:  ©  2012  General  Dynamics  C4S. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  in  millions) 

Prime  contractors:  Harris  Corporation, 

As  of 

Latest 

Percent 

Thales  Defense  &  Security 

05/2004 

12/2015 

change 

Program  office:  Aberdeen  Proving 

Research  and  development  cost 

$590.9 

$1,434.3 

142.8% 

Ground,  MD 

Procurement  cost 

$10,297.3 

$8,129.4 

-21.1% 

Funding  needed  to  complete: 

Total  program  cost 

$10,888.2 

$9,563.7 

-12.2% 

R&D:  $156.4  million 

Program  unit  cost 

$.033 

$.035 

6.4% 

Procurement:  $7,073.9  million 

Total  quantities 

328,674 

271,202 

-17.5% 

Total  funding:  $7,230.4  million 

Acquisition  cycle  time  (months) 

85 

124 

45.9% 

Procurement  quantity:  243,464 

The  JTRS  HMS  program  has  demonstrated  that 
its  critical  technologies  are  mature,  and  program 
officials  report  currently  stable  designs  for  both 
the  Rifleman  and  Manpack  radio  systems. 
Developmental  and  operational  testing  in  recent 
years  identified  deficiencies  affecting  both  radio 
systems,  which  program  officials  report  have 
since  been  resolved.  However,  the  Army  has  not 
yet  fully  verified  the  resolution  of  these  issues,  or 
the  operability  of  both  radios,  through  operational 
testing.  According  to  officials,  the  JTRS  HMS 
program  will  conduct  manufacturing  readiness 
assessments  for  both  the  Rifleman  and  Manpack 
radios  as  part  of  separate  full  rate  production 
decisions  scheduled  in  2018  and  2019, 
respectively.  Initial  fielding  of  the  Navy’s  Mobile 
User  Objective  System  (MUOS)  waveform  in 
some  radios  is  delayed  while  the  Navy  corrects 
deficiencies  it  identified  in  operational  testing. 
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Attainment  of  Product  Knowledge 

As  of  January  2017 

STATUS  CURRENT 
AT  STATUS 

I  Resources  and  requirements  match 

•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

DEVELOPMENT 

START 

O 

• 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

O 

• 

•  Complete  a  system-level  preliminary  design  review 

O 

• 

I  Product  design  is  stable 

•  Release  at  least  90  percent  of  design  drawings 

DESIGN  REVIEW 

O 

• 

•  Test  a  system-level  integrated  prototype 

• 

• 

I  Manufacturing  processes  are  mature 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

9  or  critical  processes  are  in  statistical  control 

PRODUCTION 

START 

O 

• 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

O 

6 

•  Test  a  production-representative  prototype 

• 

• 

£  Knowledge  attained  >  ■  ■  ■  Information  not  available 

O  Knowledge  not  attained  Not  applicable 
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Common  Name:  JTRS  HMS 


JTRS  HMS  Program 

Technology  and  Design  Maturity 

At  the  JTRS  HMS  program’s  development  start  in 
2004,  the  program  did  not  assess  the  maturity  of  its 
critical  technologies — an  approach  inconsistent  with 
best  practices.  Further,  the  program  had  not  fully 
matured  all  of  its  critical  technologies  by  its  2011 
production  start.  Instead,  the  program  completed 
development  of  individual  critical  technologies  in 
May  2015.  The  program  assessed  both  radio 
designs  as  stable  at  critical  design  review  in  2008, 
but  now  forecasts  new  design  changes  that  could 
undermine  this  stability. 

In  fiscal  year  2014,  developmental  testing  of  the 
Manpack  radio  identified  deficiencies  with  the 
system’s  reliability.  Initial  operational  testing  that 
same  year  flagged  suitability  shortfalls,  specifically 
related  to  excessive  weight  of  the  Manpack  units. 
Similarly,  operational  testing  of  the  Rifleman  radio  in 
fiscal  year  2014  identified  suitability  problems 
related  to  overheating  and  rapid  battery  depletion. 
Program  officials  stated  that  the  contractor  has 
redesigned  both  radio  systems  to  resolve  these 
various  problems.  However,  the  Army  has  not  yet 
completed  operational  testing  to  ensure  that  all 
deficiencies  have  been  addressed. 

Production  Maturity 

According  to  JTRS  HMS  program  officials,  the 
program  conducted  manufacturing  readiness 
assessments  for  the  Rifleman  radio  system  and 
intends  to  evaluate  the  results  in  advance  of  that 
system's  full  rate  production  decision  in  April  2018. 
They  also  stated  that  the  JTRS  HMS  program  will 
conduct  manufacturing  readiness  assessments  for 
the  Manpack  radio  as  part  of  its  separate  full  rate 
production  decision  in  May  2019.  However,  the 
program  office  has  not  yet  conducted  or  scheduled 
these  assessments,  citing  anticipated  design 
changes. 

Other  Program  Issues 

At  present,  use  of  the  MUOS  waveform — on  which 
some  Manpack  radios  will  rely — is  largely 
unavailable  to  the  warfighter  because  of  delays  with 
authorizing  its  use  in  an  operational  environment. 
Although  the  program  has  not  identified  MUOS  as  a 
critical  technology,  without  this  waveform,  affected 
Manpack  radios  are  only  able  to  communicate 
through  legacy  communications  capabilities. 


Officials  stated  that  the  authorization  delay  would 
not  impact  the  time  frame  for  the  fielding  of 
Manpack  radios  but  would  affect  the  level  of  MUOS 
availability  in  the  field.  The  Navy  conducted  initial 
operational  testing  of  the  MUOS  waveform  in 
October  and  November  2015.  Based  on  those 
results,  the  Navy  has  programmed  additional  work 
to  correct  deficiencies  along  with  an  additional 
round  of  operational  testing.  The  Army  plans  to 
leverage  this  operational  testing  to  inform  future 
fielding  decisions.  According  to  program  officials, 
changes  to  the  Manpack  radios,  necessary  for 
implementation  of  MUOS,  required  the  program  to 
extend  software  development.  JTRS  HMS  has 
successfully  completed  the  initial  set  of  MUOS 
conformance  tests  but  will  have  to  complete 
additional  conformance  tests  in  order  to  be  fully 
certified  for  use  in  the  field. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  technical  comments,  which 
were  incorporated  where  deemed  appropriate.  The 
program  office  also  disagreed  with  our 
characterization  of  past  testing  outcomes  identified 
in  this  assessment. 

GAO  Response 

Our  characterization  of  past  JTRS  HMS  testing 
outcomes  is  derived  directly  from  reports  issued  by 
the  program's  testing  authorities  within  DOD. 
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Lead  Component:  Army 


Common  Name:  M109A7FOV 


M109A7  Family  of  Vehicles  (M109A7  FOV) 


The  Army's  M109A7  FOV  system  consists  of  two 
individual  vehicles,  a  self-propelled  howitzer  (SPH), 
and  a  tracked  ammunition  carrier  that  provides 
operational  support.  The  SPH  is  a  tracked, 
aluminum  armored  vehicle  armed  with  a  155 
millimeter  cannon.  The  Ml  09A7  FOV  is  expected  to 
provide  improved  sustainability  over  the  current 
howitzer  fleet  through  the  incorporation  of  a  newly 
designed  hull;  modified  M2  Bradley  infantry  fighting 
vehicle  power  train,  suspension  system,  and  track; 
and  a  modernized  electrical  system. 


Source:  U.S.  Army. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  in  millions) 

Prime  contractor:  BAE  Systems  Land  & 

As  of 

Latest 

Percent 

Armament  L.P. 

02/2012 

12/2015 

change 

Program  office:  Warren,  Ml 

Research  and  development  cost 

$1,093.3 

$1,149.5 

5.1% 

Funding  needed  to  complete: 

Procurement  cost 

$6,160.7 

$6,255.2 

1 .5% 

R&D:  $46.2  million 

Total  program  cost 

$7,254.0 

$7,404.7 

2.1% 

Procurement:  $5,384.8  million 

Program  unit  cost 

$12,464 

$12,991 

4.2% 

Total  funding:  $5,431.0  million 

Total  quantities 

582 

570 

-2.1% 

Procurement  quantity:  503 

Acquisition  cycle  time  (months) 

118 

120 

1 .7% 

The  Ml  09A7  FOV  program  entered  low-rate  initial 
production  in  October  2013  with  its  two  critical 
technologies  mature  and  design  stable.  However, 
the  program  now  plans  costly  retrofits  to  the  first 
67  vehicles  to  improve  their  transmission  systems 
and,  for  the  howitzers,  their  cannon  tubes.  To 
date,  the  program  has  not  demonstrated  that  its 
manufacturing  processes  are  in  statistical  control, 
almost  3  years  after  production  began.  The 
program's  schedule  also  hinges  on  completing 
developmental  and  operational  testing  with  few,  if 
any,  additional  technical  discoveries.  In  the  event 
that  remaining  testing  identifies  new  deficiencies 
requiring  design  changes  or  additional  retrofitting, 
the  program  will  likely  incur  cost  increases  and 
capability  delays. 
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Attainment  of  Product  Knowledge 


As  of  January  2017 


STATUS  CURRENT 
AT  STATUS 


Demonstrate  Manufacturing  Readiness  Level  of  at  least 
9  or  critical  processes  are  in  statistical  control 


PRODUCTION 

START 


o  o 


Demonstrate  critical  processes  on  a  pilot  production  line  o  o 

Test  a  production-representative  prototype  o  • 


9  Knowledge  attained 
O  Knowledge  not  attained 


Information  not  available 
Not  applicable 


Common  Name:  M109A7  FOV 


M109A7  FOV  Program 

Technology  and  Design  Maturity 

The  M109A7  FOV  program's  two  critical 
technologies — power  pack  integration  and  the 
ceramic  bearing  of  the  generator  assembly — have 
been  assessed  as  fully  mature.  According  to 
program  officials,  the  design  is  currently  stable  with 
all  of  the  expected  drawings  released  and  a  system- 
level  integrated  prototype  demonstrated.  Despite 
this,  a  program  official  indicated  that  the  current 
M109A7  transmission  will  be  replaced  by  a  more 
efficient  transmission  in  fiscal  year  2019,  resulting  in 
retrofits  to  all  67  low-rate  production  vehicles.  The 
program  official  also  stated  that  the  SPH's  current 
steel  cannon  tube  will  be  replaced  with  a  chrome- 
plated  design  in  order  to  better  meet  its  required 
service  life.  This  retrofit  began  in  January  2016  and 
the  program  is  planning  for  611  cannon  tube 
upgrades  in  total. 

Production  Maturity 

The  program  started  low-rate  initial  production  in 
October  2013,  and  the  first  vehicle  was  delivered  in 
March  2015.  To  assess  its  production  maturity,  the 
program  compares  totals  for  expected  and  actual 
manufacturing  hours  to  assess  efficiency  gains  over 
time.  To  date,  the  program  has  not  demonstrated 
statistical  control  of  its  critical  manufacturing 
processes.  Our  best  practices  work  has  shown  that 
if  a  program's  critical  manufacturing  processes  are 
not  demonstrated  and  in  control  before  production 
begins,  it  is  at  risk  of  increased  cost  and  schedule 
delays.  Additionally,  quality  control  issues  with  the 
engine  forced  the  program  to  adjust  the  test 
schedule.  For  example,  some  vehicles  overheated. 
However,  the  program  determined  the  root  cause 
for  the  overheating  and  is  making  the  necessary 
corrections. 

Other  Program  Issues 

The  program's  current  schedule  is  predicated  on 
completing  remaining  developmental  testing  and 
initial  operational  test  and  evaluation  (IOT&E)  with 
few,  if  any,  new  discoveries  of  technical 
deficiencies.  The  M109A7  FOV's  IOT&E  was 
originally  planned  for  June  2016  has  now  been 
further  delayed  until  the  second  quarter  of  fiscal 
year  201 8,  due  to  problems  associated  with  soldiers 
being  exposed  to  toxic  fumes  during  a  test  event  in 
October  2016.  Root-cause  analysis  indicated  a 
combination  of  breech  malfunctions,  improper  crew 


procedures,  and  improper  maintenance  caused 
toxic  fumes  in  certain  guns  to  rise  to  dangerous 
levels.  The  program  has  addressed  hardware 
issues  and  added  additional  training,  to  address  the 
factors  that  led  to  this  dangerous  situation.  The  low- 
rate  initial  production  is  now  being  extended  through 
fiscal  year  2017  and  the  full-rate  production  decision 
was  planned  for  December  2017  is  now  scheduled 
for  June  2018.  Additional  schedule  delays  and  cost 
increases  could  result  if  additional  deficiencies  are 
found  during  IOT&E  that  affect  the  M109A7  FOV's 
ability  to  be  found  operationally  effective  and 
suitable  and,  subsequently,  require  design  changes 
or  retrofits.  For  example,  based  on  previous 
developmental  testing  results,  program  officials 
redesigned  howitzer  hardware  and  software  to 
improve  maximum  rate  of  fire  performance. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment ,  the 
program  office  provided  technical  comments,  which 
were  incorporated  where  deemed  appropriate. 
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Lead  Component:  Army 


Common  Name:  PAC-3  MSE 


Patriot  Advanced  Capability-3  Missile  Segment  Enhancement  (PAC-3  MSE) 

The  Army's  PAC-3  MSE  is  a  surface-to-air  missile 
designed  to  defeat  tactical  ballistic  missiles  and 
other  aerial  threats.  The  MSE  is  the  latest  version  of 
PAC-3  missiles  integrated  into  the  PATRIOT 
system,  which  includes  radars,  launchers,  and  a 
command  and  control  system.  The  PAC-3  MSE 
improves  upon  earlier  PAC-3  variants  and  is 
expected  to  provide  a  more  lethal  interceptor  with 
expanded  range  and  accuracy  against  complex 
threats. 


Source:  U.S.  Army. 
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Program  Essentials 

Prime  contractor:  Lockheed  Martin 
Program  office:  Redstone  Arsenal,  AL 
Funding  needed  to  complete: 

R&D:  $0.0  million 
Procurement:  $3,875.3  million 
Total  funding:  $3,884.9  million 
Procurement  quantity:  813 


Program  Performance  (fiscal  year  2017  dollars  in  millions) 


Research  and  development  cost 

As  of 
08/2004 

$585.6 

Latest 

10/2016 

$971.6 

Percent 

change 

65.9% 

Procurement  cost 

$7,318.9 

$5,556.6 

-24.1% 

Total  program  cost 

$7,904.6 

$6,537.8 

-17.3% 

Program  unit  cost 

$5,173 

$5,811 

12.3% 

Total  quantities 

1,528 

1,125 

-26.4% 

Acquisition  cycle  time  (months) 

NA 

143 

NA 

Table  note:  Change  in  acquisition  cycle  time  could  not  be  calculated  as  initial  estimates  did  not 
include  an  initial  operational  capability  date. 


The  PAC-3  MSE  program  entered  production  in 
2014  with  mature  technologies  and  a  stable 
design.  Two  years  after  production  start,  the 
program  successfully  demonstrated  its 
manufacturing  processes  were  under  control.  The 
program  achieved  initial  operational  capability  in 
2016  ahead  of  schedule,  which  program  officials 
attribute  to  early  missile  deliveries  and  training 
efficiencies.  The  Army  has  also  successfully 
completed  three  flight  tests  of  the  missile  during 
developmental  testing  for  various  PATRIOT 
system  software  and  hardware  upgrades.  While 
the  program  has  made  some  progress  finalizing 
the  terms  of  its  2015  missile  production 
undefinitized  contract  action,  the  Army  does  not 
expect  to  complete  this  process  until  nearly  a  year 
and  a  half  after  the  issuance  of  the  action,  which 
may  place  the  program  at  risk  of  cost  growth. 
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Attainment  of  Product  Knowledge 

As  of  January  2017 

STATUS 

AT 

CURRENT  1 
STATUS  1 
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DEVELOPMENT 
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START 
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final  form,  fit,  and  function  within  a  relevant  environment 
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1  Product  design  is  stable 

DESIGN  REVIEW 

•  Release  at  least  90  percent  of  design  drawings 

O 

• 

•  Test  a  system-level  integrated  prototype 

O 

• 

Manufacturing  processes  are  mature 


PRODUCTION 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 


9  or  critical  processes  are  in  statistical  control 

O 

• 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

• 

• 

•  Test  a  production-representative  prototype 

• 

• 

0  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

O  Knowledge  not  attained  Not  applicable 
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Common  Name:  PAC-3  MSE 


PAC-3  MSE  Program 

Technology  and  Design  Maturity 

The  PAC-3  MSE  program  entered  production  in 
March  2014  with  a  stable  design  and  all  four  of  its 
critical  technologies  mature.  Although  the  PAC-3 
MSE  has  components  that  are  90  percent  common 
with  an  earlier  PAC-3  variant,  four  unique 
technologies  required  development — a  dual  pulse 
solid  rocket  motor,  thermal  batteries,  an  ignition 
safety  device,  and  insensitive  munitions  to  prevent 
inadvertent  launch  or  detonation. 

Production  Maturity 

The  PAC-3  MSE  program  has  demonstrated  its 
manufacturing  processes  are  in  control.  However, 
this  demonstration  did  not  occur  until  over  2  years 
after  production  start,  which  is  inconsistent  with  best 
practices.  PAC-3  MSE  began  production  in  March 
2014  by  demonstrating  that  all  materials, 
manpower,  tooling,  and  facilities  were  proven  on  a 
pilot  production  line  and  were  available  to  meet  the 
low  rate  production  schedule,  as  recommended  by 
DOD  guidance.  However,  according  to  best 
practices,  programs  should  demonstrate 
manufacturing  processes  to  be  in  statistical  control 
prior  to  production  start.  Specifically,  critical 
processes  should  be  repeatable,  sustainable,  and 
consistently  producing  parts  within  the  quality 
standards. 

Other  Program  Issues 

The  PAC-3  MSE  program  achieved  initial 
operational  capability  in  July  201 6 — about  5  months 
ahead  of  schedule — by  equipping  a  PATRIOT 
battalion  with  48  missiles.  Program  officials 
attributed  the  early  completion  of  this  milestone  to 
faster  than  expected  missile  deliveries  from  the 
prime  contractor  and  synergies  with  existing  training 
for  PATRIOT  battalions. 

In  2016,  the  program  successfully  completed  three 
flight  tests  for  the  PAC-3  MSE  during  developmental 
testing  for  various  PATRIOT  system  software  and 
hardware  upgrades.  The  PATRIOT  system  required 
these  upgrades  to  fully  test  and  utilize  the  missile's 
capabilities.  Future  results  from  system-level 
operational  testing  of  the  missile  with  these 
upgrades,  which  began  in  fiscal  year  2016,  will  be 
used  to  support  a  PAC-3  MSE  full-rate  production 
decision,  which  is  currently  planned  for  2017. 


The  PAC-3  MSE  program  continues  to  encounter 
delays  in  definitizing  its  production  undefinitized 
contract  actions.  Last  year,  we  reported  that 
completion  of  the  final  contract  pricing  and  terms  for 
the  program's  2014  production  contract  was 
delayed  over  a  year  beyond  its  expected  date.  We 
previously  reported  delays  in  finalizing  this  type  of 
contract  places  the  program  at  risk  of  cost  growth  as 
the  government  normally  reimburses  contractors  for 
all  allowable  costs  incurred  during  the  undefinitized 
period,  giving  contractors  little  incentive  to  control 
costs.  The  program  is  also  facing  delays  in  finalizing 
terms  for  its  2015  production  contract.  Although  the 
program  made  some  progress  by  determining 
nearly  85  percent  of  the  contract  terms  within  180 
days,  officials  do  not  expect  to  fully  complete  this 
process  until  December  2016 — nearly  a  year  and  a 
half  after  the  issuance  of  the  action. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  technical  comments,  which 
were  incorporated  where  appropriate. 
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Lead  Component:  Army 


Common  Name:  WIN-T  Inc  2 


Warfighter  Information  Network-Tactical  Increment  2  (WIN-T  Inc  2) 


WIN-T  is  the  Army's  high-speed  and  high-capacity 
tactical  communications  network.  It  connects  units 
with  higher  levels  of  command  and  is  being  fielded 
in  three  increments.  The  Army  completed  fielding  of 
the  first  increment  in  2012  and  ended  the 
development  of  the  third  increment  in  201 6. 
Together,  those  increments  provided  data,  voice, 
and  video  networking  capability,  along  with  critical 
software  upgrades.  The  Army  expects  the  second 
increment,  Increment  2,  to  provide  required 
networking  on-the-move  capability.  We  assessed 
Increment  2. 


Source:  U.  S.  Army. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  in  millions) 

Prime  contractor:  General  Dynamics  C4 

As  of 

Latest 

Percent 

Systems,  Inc. 

10/2007 

09/2016 

change 

Program  office:  Aberdeen  Proving 

Research  and  development  cost 

$258.8 

$334.8 

29.4% 

Ground,  MD 

Procurement  cost 

$3,763.4 

$10,875.1 

189.0% 

Funding  needed  to  complete: 

Total  program  cost 

$4,022.2 

$11,209.9 

178.7% 

R&D:  $44.2  million 

Program  unit  cost 

$2,125 

$3,005 

41 .4% 

Procurement:  $7,525.0  million 

Total  quantities 

1,893 

3,730 

97.0% 

Total  funding:  $7,569.2  million 

Acquisition  cycle  time  (months) 

50 

74 

48.0% 

Procurement  quantity:  2,137 

WIN-T  Increment  2  entered  production  in  March 
2010  with  its  critical  technologies  mature.  The 
program  does  not  track  the  metric  we  use  to 
measure  design  stability  as  WIN-T  is  primarily  an 
information  technology  integration  effort.  Instead, 
design  performance  is  measured  through  testing 
designed  to  demonstrate  performance  at 
increasing  levels  of  system  integration.  The 
program  has  met  its  revised  reliability  criteria  and 
has  improved  its  cyber-defense  capabilities.  In 
January  2015,  the  Army  reported  that  the 
program  had  breached  a  statutory  unit  cost 
growth  threshold.  Program  officials  stated  that 
they  are  undertaking  steps  to  reduce  future  unit 
costs  below  this  threshold  level. 
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Attainment  of  Product  Knowledge 


As  of  January  2017 


Resources  and  requirements  match 
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Common  Name:  WIN-T  Inc  2 


WIN-T  Inc  2  Program 

Technology,  Design,  and  Production  Maturity 

WIN-T  Increment  2  entered  development  in  June 
2007  with  15  critical  technologies.  Of  these,  3  were 
mature,  4  were  approaching  maturity,  and  8  were 
immature  or  not  assessed  at  development  start.  By 
the  program's  March  2010  production  start,  all  15 
critical  technologies  were  mature.  The  program 
does  not  track  the  metric  we  use  to  measure  design 
stability — the  number  of  releasable  drawings — as 
WIN-T  is  primarily  an  information  technology 
integration  effort.  Instead,  design  performance  is 
measured  through  a  series  of  tests  designed  to 
demonstrate  performance  at  increasing  levels  of 
system  integration.  The  program  began  testing  a 
production  representative  prototype  in  March  2011 
and  assessed  that  its  manufacturing  processes 
were  within  statistical  control  in  2012,  both  after 
production  began. 

Other  Program  Issues 

The  program  made  progress  against  performance 
and  reliability  objectives — issues  evident  during 
earlier  rounds  of  operational  testing.  This  was 
achieved,  in  part,  due  to  a  relaxing  of  the  reliability 
objectives  as  part  of  an  update  to  the  program's 
capability  production  document.  The  prior 
operational  reliability  standard  of  90  percent — or 
probability  of  mission  success  without  an  essential 
functional  failure — was  deemed  "excessively  high 
when  compared  to  predecessor,  current,  and 
analogous  systems  during  similar  periods  in  their 
respective  life  cycles."  The  operational  reliability 
threshold  was  revised  to  80  percent,  and  the 
program  met  this  threshold  during  prior  testing. 

While  the  program  continues  to  carry  a  high  risk 
regarding  its  defensive  capabilities  against  cyber¬ 
attacks,  it  is  also  making  progress  in  mitigating 
these  deficiencies.  According  to  program  officials,  in 
order  to  reduce  this  risk,  the  program  is  consistently 
updating  its  cyber  defenses,  testing  them  against 
both  internal  and  independent  third-party  attack 
teams,  and  then  refining  its  cyber  defenses  based 
on  the  results.  For  example,  the  program  is  working 
with  an  independent  team  at  Johns  Hopkins 
University  that  attempts  to  hack  the  program's 
system  and  must  also  defend  against  other  U.S. 
Army  cyber  attackers  as  part  of  the  program's 
network  integration  tests.  Program  officials  stated 
that  since  fiscal  year  2015,  the  program  has 


improved  its  cyber  defenses,  but  expects  to 
continue  to  improve  over  time  and  reduce  the  risk 
level.  However,  officials  also  noted  that  given  the 
nature  of  cyber-attacks  and  cyber  defense,  the  risk 
would  likely  remain  throughout  the  life  of  the 
program  as  an  issue  that  will  always  need  attention. 

The  program  continues  to  track  cost  growth  as  a 
concern  following  the  January  2015  announcement 
that  the  program  had  breached  a  statutory 
significant  unit  cost  threshold.  Program  officials 
stated  that  the  program  continues  to  reduce  unit 
costs,  which  should  bring  down  average  unit  costs 
over  time.  In  addition,  officials  stated  that  the 
program  is  working  to  reduce  costs  further  by 
decreasing  the  complexity  of  the  hardware — both  in 
terms  of  size  and  number  of  components — and  by 
employing  better  buying  power  initiatives.  With 
these  efforts,  officials  are  targeting  further  reduced 
costs  in  the  future. 

Program  Office  Comments 

Program  officials  stated  that  WIN-T  is  currently 
fielded  to  14  Brigade  Combat  Teams,  7  Division 
Headquarters,  and  the  U.S.  Army  Signal  School, 
Fort  Gordon,  GA.  Officials  noted  the  program 
remains  on  track  to  field  additional  units  as  directed 
by  the  Army  at  a  rate  of  approximately  two  per  year. 
Officials  also  reported  the  program  continues  to  use 
soldier  feedback  from  theater  and  test  results  to 
improve  WIN-T  Increment  2  capabilities  with  the 
primary  focus  being  on  simplifying  the  operations 
and  maintenance  of  the  equipment.  Finally,  officials 
reported  the  program  office  has  developed  new 
vehicle-based  capabilities  to  meet  emerging  Army 
expeditionary  and  transportability  needs  and  will 
test  these  capabilities  at  an  integration  test  event  in 
summer  2017. 
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Common  Name:  ITEP 

Improved  Turbine  Engine  Program  (ITEP) 


The  Army’s  Improved  Turbine  Engine  Program  (ITEP)  is 
developing  a  replacement  engine  for  the  Black  Hawk 
and  Apache  helicopter  fleets.  The  new  engine  is 
designed  for  increased  power,  performance,  and  fuel 
efficiency;  enhanced  reliability;  increased  service  life; 
and  a  lower  maintenance  burden.  The  Army  plans  to 
begin  fielding  these  engines  in  fiscal  year  2026. 


Source:  U.S.  Army 


Current  Status 

In  August  2016,  the  Assistant  Secretary  of  the  Army,  for  Acquisition,  Logistics,  and  Technology,  approved 
ITEP’s  entry  into  the  technology  maturation  and  risk  reduction  phase.  The  Army  awarded  two  fixed-price 
incentive  (firm  target)  technology  maturation  and  risk  reduction  contracts  totaling  $254.7.million  to  General 
Electric  and  Advanced  Turbine  Engine  Corporation  with  the  goals  of  planning,  designing,  and  developing  the 
engine  through  the  preliminary  design  review.  Each  contractor  will  hold  a  separate  preliminary  design  review 
in  the  second  quarter  of  fiscal  year  2018  prior  to  entry  into  system  development.  The  Army  intends  to  award 
a  cost-plus-incentive-fee  contract  for  system  development  with  competition  limited  to  the  two  contractors 
awarded  preliminary  design  contracts. 

An  initial  technology  readiness  assessment  completed  in  October  2014  identified  three  critical  technologies — 
advanced  inlet  particle  separator,  compressor/advanced  aerodynamics,  and  hybrid  bearings — as 
approaching  full  maturity.  A  second  assessment  in  support  of  system  development  start,  planned  for  January 
2017,  will  provide  updated  information  on  each  contractor’s  critical  technologies. 

In  2015,  Sikorsky  Aircraft  Corporation  and  Boeing  initiated  trade  studies  to  evaluate  potential  approaches  for 
integrating  ITEP  hardware,  electrical  systems,  and  software  into  Black  Hawk  and  Apache  helicopter 
airframes.  The  Army  is  investing  approximately  $24.4  million  to  complete  these  trade  studies  and  plans  to  rely 
on  them  to  define  a  system  integration  approach  that  balances  cost  and  schedule  considerations,  and 
required  airframe  modifications  and  component  constraints.  According  to  the  program  office,  the  trade  studies 
will  also  identify  system  integration  risks  and  facilitate  development  of  mitigation  strategies  prior  to  the  start  of 
system  development. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2017  dollars): 

Total  program:  $10,445.89  million 

Research  and  development:  $2,417.79  million 

Procurement:  $8,028.10  million 

Quantity:  74  (development),  6,215  (procurement) 

Next  Major  Program  Event:  System  development  start,  third  quarter  fiscal  year  2018 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  program  office  provided 
technical  comments,  which  were  incorporated  where  deemed  appropriate. 
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Long  Range  Precision  Fires  (LRPF) 


Common  Name:  LRPF 


Long  Range  Precision  Fires  (LRPF)  are  part  of  a  family 
of  ballistic  missiles  designed  to  attack  area  and  point 
targets  to  ranges  of  300  kilometers  and  beyond.  Each 
LRPF  launch  pod  missile  container  is  intended  to  hold  a 
minimum  of  one  and  up  to  four  missiles.  LRPF  is 
compatible  with  existing  Ml 42  and  M270  rocket  launch 
Systems.  LRPF  is  intended  to  replace  the  Army  Tactical 
Missile  System  (ATACMS)  and  comply  with  statutory 
requirements  for  insensitive  munitions  and  DOD  policy  on 
cluster  munitions. 


Source:  U.S.  Army 


Current  Status 

LRPF  is  scheduled  to  enter  technology  development  in  January  201 7.  Originally  scheduled  for  January  201 6, 
program  officials  told  us  this  year-long  delay  follows  a  review  of  the  analysis  of  alternatives  and 
reconsideration  of  the  LRPF  range  requirement  and  launch  pod  capacity.  LRPF  program  officials  maintain  this 
will  not  affect  the  overall  program  schedule  as  long  as  technology  development  begins  by  March  201 7.  The 
start  of  system  development  was  delayed  from  2020  to  2021 . 

The  Army  has  identified  two  critical  technologies  for  LRPF — the  rocket  motor  and  warhead.  The  Army  is 
relying  on  these  two  technologies  to  provide  capabilities  that  allow  LRPF  to  meet  its  minimum  range 
requirement  and  limit  unexploded  ordnance  to  levels  compliant  with  DOD’s  policy  on  cluster  munitions, 
respectively.  The  program  recently  increased  the  number  of  test  missiles  it  plans  to  procure  from  6  to  8 
during  technology  development,  and  40  during  system  development.  The  LRPF  program  has  an  approved 
test  and  evaluation  master  plan.  An  independent  cost  estimate,  which  will  inform  the  decision  to  enter 
technology  development,  is  underdevelopment. 

LRPF  will  employ  competitive  prototyping  prior  to  the  start  of  system  development.  In  2016,  the  Army  awarded 
cost-plus-fixed-fee  contracts  to  Lockheed  Martin  and  Raytheon  to  begin  developing  LRPF  missile  designs. 
This  work  will  inform  the  program’s  technology  readiness  assessment,  which  it  plans  to  complete  prior  to 
system  development.  Program  officials  expect  to  select  a  single  contractor  to  execute  LRPF  system 
development. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2017  dollars): 

Total  program:  $2,901 .4  million 

Research  and  development:  $771 .7  million 

Procurement:  $2,129.7  million 

Quantity:  48  (development),  2,422  (procurement) 

Next  Major  Program  Event:  Start  of  technology  development,  January  2017 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  program  office  provided 
technical  comments,  which  were  incorporated  where  deemed  appropriate. 
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Navy  and  U.S.  Marine  Corps  Service  Summary 


We  completed  individual  assessments  on  23  of  the  Navy  and  Marine  Corps’  41  current  and  future  major  defense 
acquisition  programs.  Of  these  23  programs,  17  are,  for  the  most  part,  in  system  development  or  early 
production;  3  are  future  programs  DOD  expects  to  enter  system  development  in  the  next  few  years;  2  are 
programs  that  are  well  into  production,  but  planning  to  introduce  new  increments  of  capability;  and  1  is  the  Navy’s 
Frigate  initiative,  which  is  currently  part  of  the  Navy’s  Littoral  Combat  Ship  (LCS)  program,  but  is  expected  to  be 
restructured  into  its  own  separate  program  soon.  We  found  the  Navy  and  Marine  Corps  currently  estimate  a  need 
of  $236  billion  in  funding  to  complete  the  acquisition  of  these  23  programs.  In  addition,  we  compared  these  23 
programs’  first  full  estimates  of  cost  and  schedule,  as  available,  with  their  current  estimates  and  found: 

•  net  cost  growth  totals  approximately  $24  billion,  all  of  which  occurred  more  than  5  years  ago,  and 

*  program  schedule  delays  average  35  months. 

Further,  1  of  the  23  programs — LHA  6  America  Class  Amphibious  Assault  Ship — completed  all  the  activities 
associated  with  the  applicable  knowledge  based  best  practices  we  assess,  although  these  activities  were  not  fully 
complete  at  the  time  the  program  reached  these  key  knowledge  points. 


Acquisition  Phase  and  Size  of  the  Currently  Estimated  Acquisition  Cost  Summary  of  Knowledge  Attained  to  Date  for 

23  Programs  Assessed  for  the  23  Programs  Assessed  Programs  Beyond  System  Development  Start 


Production 


Fiscal  year  201 7  dollars  in  billions 


MQ-8 
Fire  Scout 

•  • 

DDG 

1000 


LHA  6 


CVN  78 


LCS 

Packages 


G/ATOR 


SSC 


MQ-4C 

Triton 


OASuW 
Inc  1 


System  development 


•  Aamdr 
JPALS 

A 


VH-92A. 


NGJ  Increment  1 


Technology  development 


r 

1 

r 

ISBN  826 

* 

A 

T-AO  205 


$14.8 

Programs 
in  system 
development 


Programs  in 
production 


Cost  and  Schedule  Growth  on  14  Programs 
in  the  Current  Portfolio 

Fiscal  year  2017  dollars  in  billions 

$24.2 


Research  and  Procurement  Total 

development  costs  acquisition 

costs  costs 


□  Growth  from  first 
full  estimate 


□ 


First  full  estimate 


•  Cost  growth  of  more  than  15  percent  and/or 
schedule  delays  of  more  than  6  months 

•  Cost  growth  of  1 5  percent  or  less  and  schedule 
delays  of  6  months  or  less 

No  first  full  estimate  available 


Note:  Bubble  size  is  based  on  each  program's  currently  estimated 
future  funding  needed. 


Average 
acquisition 
cycle  time 

(in  months) 


Note:  For  acquisition  cycle  time  only  13  programs  were  assessed 
as  not  all  programs  contained  sufficient  information  within  their 
first  full  estimates  to  determine  acquisition  cycle  time.  In  addition 
to  research  and  development  and  procurement  costs,  total 
acquisition  cost  includes  acquisition  related  operation  and 
maintenance  and  system-specific  military  construction  costs. 
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processes  are 
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o  o 
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o  • 

o  o 

KP  3  in  future 

a  All  applicable  knowledge  practices 
w  completed 

/  n  One  or  more  applicable  knowledge 
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|  Knowledge  practice  is  not  applicable 

Information  not  available  for  knowledge 
practice 


Source:  GAO  analysis  of  DOD  data.  |  GAO-1 7-333SP 
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Navy  and  Marine  Corps  Program  Assessments 

2-page  assessments 

Page  Number 

Air  and  Missile  Defense  Radar  (AMDR) 

89 

Amphibious  Combat  Vehicle  (ACV) 

91 

CH-53K  Heavy  Lift  Replacement  Helicopter  (CH-53K) 

93 

CVN  78  Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier  (CVN  78) 

95 

DDG  1000  Zumwalt  Class  Destroyer  (DDG  1000) 

97 

Ground/Air  Task  Oriented  Radar  (G/ATOR) 

99 

Joint  Precision  Approach  and  Landing  System  (JPALS) 

101 

LHA  6  America  Class  Amphibious  Assault  Ship  (LHA  6) 

103 

Littoral  Combat  Ship  (LCS) 

105 

Littoral  Combat  Ship  -  Mission  Modules  (LCS  Packages) 

107 

MQ-4C  Triton  Unmanned  Aircraft  System  (MQ-4C  Triton) 

109 

MQ-8  Fire  Scout  (MQ-8  Fire  Scout) 

111 

Next  Generation  Jammer  Increment  1  (NGJ  Increment  1) 

113 

Offensive  Anti-Surface  Warfare  Increment  1  (OASuW  Inc  1) 

115 

SSBN  826  Columbia  Class  Ballistic  Missile  Submarine  (SSBN  826) 

117 

Ship  to  Shore  Connector  Amphibious  Craft  (SSC) 

119 

VH-92A  Presidential  Helicopter  Replacement  Program  (VH-92A) 

121 

1-page  assessments 


Amphibious  Ship  Replacement  (LX(R)) 

123 

DDG  51  Flight  III  Destroyer  (DDG  51  Flight  III) 

124 

John  Lewis  Class  Fleet  Replenishment  Oiler  (T-AO  205) 

125 

MQ-25  Unmanned  Aircraft  System  (MQ-25) 

126 

Navy  Frigate  (Frigate) 

127 

P-8A  Increment  3  (P-8A) 

128 
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Lead  Component:  Navy 


Common  Name:  AMDR 


Air  and  Missile  Defense  Radar  (AMDR) 


The  Navy's  AMDR  is  a  next-generation  radar 
program  supporting  surface  warfare  and  integrated 
air  and  missile  defense.  AMDR  is  developing  an  S- 
band  radar — known  as  SPY-6 — that  is  expected  to 
have  increased  sensitivity  for  long-range  detection 
to  improve  ballistic  missile  defense  against 
advanced  threats.  The  program  is  also  developing  a 
radar  suite  controller  to  provide  radar  resource 
management  and  coordination,  and  to  interface  with 
an  upgraded  Aegis  combat  system  to  provide 
integrated  air  and  missile  defense  for  DDG  51  Flight 
III  destroyers. 


Source:  ©  2015  Raytheon  Company. 
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▲ 

▲ 
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Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  in  millions) 

Prime  contractor:  Raytheon 

As  of 

Latest 

Percent 

Program  office:  Washington,  DC 

10/2013 

12/2015 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$1,966.4 

$1,955.2 

-0.6% 

R&D:  $468.7  million 

Procurement  cost 

$4,066.9 

$3,440.2 

-15.4% 

Procurement:  $3,191 .6  million 

Total  program  cost 

$6,063.7 

$5,425.6 

-10.5% 

Total  funding:  $3,660.2  million 

Program  unit  cost 

$275,625 

$246,619 

-10.5% 

Procurement  quantity:  22 

Total  quantities 

22 

22 

0.0% 

Acquisition  cycle  time  (months) 

155 

155 

0.0% 

AMDR  completed  its  critical  design  review  in  April 
2015  with  four  critical  technologies  nearing 
maturity  and  its  design  stable.  The  program  has 
largely  completed  its  four  planned  software 
builds,  with  the  last  build's  completion  expected 
by  April  2017.  In  June  2016,  a  full-scale, 
production-representative  SPY-6  radar  array  was 
installed  at  the  Navy's  Pacific  Missile  Range 
Facility  for  land-based  testing.  Testing  of  the  radar 
system  at  sea  is  not  planned  until  2020,  and  initial 
operational  test  and  evaluation  is  expected  in 
2023.  The  program's  test  and  evaluation  master 
plan  remains  unapproved  by  DOD's  Director, 
Operational  Test  and  Evaluation  (DOT&E)  due  to 
concerns  that  the  Navy’s  proposed  testing 
approach  would  not  perform  testing  under 
realistic  operational  conditions.  The  test  plan  may 
not  be  approved  before  the  AMDR  low-rate  initial 
production  decision  in  September  2017. 
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Attainment  of  Product  Knowledge 


As  of  January  2017 

STATUS  CURRENT 
AT  STATUS 

Resources  and  requirements  match 

DEVELOPMENT 

•  Demonstrate  all  critical  technologies  are  very  close  to 

START 

A  A 

final  form,  fit,  and  function  within  a  relevant  environment 

•  • 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 

o  o 

function  within  a  realistic  environment 

•  Complete  a  system-level  preliminary  design  review 

o  • 

1  Product  design  is  stable 

DESIGN  REVIEW 

•  Release  at  least  90  percent  of  design  drawings 

O  • 

•  Test  a  system-level  integrated  prototype 

•  • 

1  Manufacturing  processes  are  mature 

PRODUCTION 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

START 

9  or  critical  processes  are  in  statistical  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 

0  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

O  Knowledge  not  attained  Not  applicable 

Common  Name:  AMDR 


AMDR  Program 

Technology  Maturity 

AMDR's  four  critical  technologies — digital 
beamforming,  transmit/receive  modules,  multi¬ 
mission  scheduling  and  discrimination  software,  and 
distributed  receivers/exciters — are  nearing  maturity. 
The  program  is  expected  to  deliver  its  first  radar  for 
installation  on  the  lead  DDG  51  Flight  III  ship  in 
early  2020.  To  support  initial  integration  between  the 
radar  and  the  combat  system,  the  AMDR  contractor 
developed  and  delivered  SPY-6  simulator  and 
emulator  capabilities  to  help  inform  the  program's 
knowledge  of  the  radar  and  Aegis  combat  system 
interface  performance  prior  to  a  6-month  risk 
reduction  test  period  planned  for  the  second  half  of 
fiscal  year  2017.  Additionally,  the  contractor  built 
and  tested  a  full-scale,  single-face,  14-foot  S-band 
radar  array.  In  June  2016,  this  production- 
representative  array  was  delivered  and  installed  at 
the  Navy's  Pacific  Missile  Range  Facility  in  Plawaii 
for  live  testing  in  a  more  representative 
environment.  This  testing  is  expected  to  reduce 
technical  risk  for  the  radar  and  help  inform  a  low- 
rate  initial  production  decision  in  September  2017. 

The  AMDR  program's  software  has  been  developed 
in  four  builds  using  an  approach  that  includes 
upfront  requirements  and  architecture  analysis  for 
each  build,  as  well  as  continuous  integration  of  new 
software  and  automated  testing  to  ensure 
functionality  and  performance.  This  includes 
aligning  software  features  to  test  events  to  ensure 
timely  software  completion  and  delivery  to  support 
dry  runs  and  tests.  The  first  two  builds  developed 
basic  infrastructure,  anti-air  warfare,  and  ballistic 
missile  defense  capabilities.  The  third  and  fourth 
provide  the  full  extent  of  radar  capabilities,  including 
debris  detection  and  mitigation  and  advanced 
discrimination  of  missile  threats.  As  of  December 
201 6,  the  fourth  build  was  80  percent  complete,  with 
completion  planned  by  April  201 7 — about  half  a 
year  later  than  previously  planned.  The  Navy  also 
plans  to  upgrade  the  combat  system  for  integration 
with  the  SPY-6  radar,  which  will  require  significant 
software  development  for  the  interface  between  the 
radar  and  the  combat  management  system.  These 
software  builds  are  expected  to  be  completed  in 
fiscal  year  2021. 


Design  and  Production  Maturity 

In  April  2015,  the  program  office  completed  a  critical 
design  review,  with  100  percent  of  design  drawings 
finalized.  The  design  has  remained  stable  as  the 
program  moves  toward  its  initial  production 
decision.  However,  because  the  decision  to  begin 
low-rate  initial  production  will  be  made  prior  to 
demonstrating  technology  maturity  at  sea  and 
before  combat  system  integration  and  test,  design 
stability  remains  a  risk.  Any  design  issues  identified 
through  testing  the  radar  at  sea  and  with  the  combat 
system  will  need  to  be  addressed  during  SPY-6 
production.  The  program  office  identified  four  key 
product  characteristics  that  will  be  closely  monitored 
during  manufacturing.  The  characteristics  are 
associated  with  the  structural  features  of  the  radar 
and  elements  of  the  transmit/receive  modules  and 
beamforming  technologies. 

Other  Program  Issues 

In  201 3,  DOT&E  disapproved  AMDR's  test  and 
evaluation  master  plan  due  to  concerns  the  Navy's 
proposed  testing  approach  would  not  provide 
realistic  operational  conditions.  A  senior  DOT&E 
official  noted  the  concern  that  the  Navy  has  not 
involved  DOT&E  in  efforts  to  update  the  test  plan 
and  anticipates  that,  without  DOT&E  involvement, 
the  plan  is  likely  to  remain  unapproved  by  DOT&E 
when  the  program  reaches  its  September  2017 
production  decision. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  stated  the  SPY-6  testing  it  completed 
at  the  Pacific  Missile  Range  Facility  validated  the 
system  performance  previously  measured  in  testing 
at  the  contractor’s  facility,  allowing  the  program  to 
procure  long  lead  material  for  the  first  DDG  51  Flight 
III  in  December  2016.  Upcoming  live  testing  of 
several  systems  is  expected  to  demonstrate  the 
advanced  features  and  unprecedented  capability  of 
this  radar.  The  program  reports  being  on  track  to 
provide  this  much-needed  capability  to  the 
warfighter. 
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Lead  Component:  Navy 


Common  Name:  ACV 


Amphibious  Combat  Vehicle  (ACV) 


The  Marine  Corps'  ACV  is  the  successor  program  to 
the  canceled  Expeditionary  Fighting  Vehicle  (EFV). 
The  ACV  is  intended  to  transport  Marines  from  ship 
to  shore  and  provide  improved  mobility  and  high 
levels  of  protection.  The  acquisition  approach  calls 
for  three  increments  of  development  (1.1,  1.2,  and 
2.0)  and  leverages  work  accomplished  under  the 
EFV  program.  The  Marine  Corps  expects  later  ACV 
increments  to  have  improved  amphibious  capability 
over  the  first  increment,  ACV  Increment  1.1.  We 
assessed  ACV  Increment  1.1. 


Source:  U.S.  Marine  Corps. 
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Program  Essentials 

Prime  contractor:  BAE  Systems  Land  & 
Armaments  LP,  Science  Applications 
International  Corporation 
Program  office:  Stafford,  VA 
Funding  needed  to  complete: 

R&D:  $382.0  million 
Procurement:  $1 ,057.8  million 
Total  funding:  $1 ,477.7  million 
Procurement  quantity:  204 


Program  Performance  (fiscal  year  2017  dollars  in  millions) 


As  of 

Latest 

08/2016 

Percent 

change 

Research  and  development  cost 

NA 

$796.3 

NA 

Procurement  cost 

NA 

$1,057.8 

NA 

Total  program  cost 

NA 

$1,903.4 

NA 

Program  unit  cost 

NA 

$7,931 

NA 

Total  quantities 

NA 

240 

NA 

Acquisition  cycle  time  (months) 

NA 

57 

NA 

The  ACV  program  completed  a  combined 
preliminary/critical  design  review  in  October  2016 
with  mature  critical  technologies  and  stable 
designs  from  its  two  contractor  teams.  The 
program  includes  two  critical  technologies,  one  of 
which  is  currently  in  use  on  an  existing  platform. 
The  other  technology,  the  Remote  Weapon 
Station,  completed  testing  in  June  2016, 
demonstrating  it  can  operate  in  a  marine 
environment.  According  to  the  program  office, 
both  contractors  released  90  percent  of  their 
drawings  at  the  design  review  and  are  developing 
prototypes.  The  program  schedule  includes 
concurrent  testing  and  production  that  place  ACV 
at  risk  for  costly  retrofits  as  knowledge  is  gained 
in  testing.  Plans  are  in  place  for  corrective  action 
periods  to  implement  modifications  stemming 
from  system-level  developmental  testing.  The 
Marine  Corps  plans  to  select  a  single  vendor  in 
201 8  to  produce  the  ACV. 


Attainment  of  Product  Knowledge 


As  of  January  2017 


Resources  and  requirements  match 


STATUS  CURRENT 
AT  STATUS 


•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

DEVELOPMENT 

START 

• 

• 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

o 

• 

•  Complete  a  system-level  preliminary  design  review 

o 

• 

Product  design  is  stable 

•  Release  at  least  90  percent  of  design  drawings 

DESIGN  REVIEW 

• 

• 

•  Test  a  system-level  integrated  prototype 

o 

o 

Manufacturing  processes  are  mature 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

PRODUCTION 

START 

9  or  critical  processes  are  in  statistical  control 
Demonstrate  critical  processes  on  a  pilot  production  line 
Test  a  production-representative  prototype 


^  Knowledge  attained 
O  Knowledge  not  attained 


Information  not  available 
Not  applicable 
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Common  Name:  ACV 


ACV  Program 

Technology  Maturity 

ACV  entered  system  development  in  2015  with  one 
mature  critical  technology  and  another  critical 
technology  approaching  maturity.  The  program  now 
assesses  both  technologies  as  fully  mature.  One 
technology,  the  Driver's  Vision  Enhancement, 
assists  the  ACV  driver  in  safely  operating  the 
vehicle  with  a  closed  hatch  during  both  the  day  and 
night  and  is  currently  used  on  the  Assault 
Amphibious  Vehicle.  The  other  critical  technology, 
the  Remote  Weapon  Station  (RWS),  is  a  stabilized, 
remote  operated,  weapon  station  designed  to 
operate  in  the  maritime  environment.  The  RWS  is 
government-furnished  equipment  that  will  be 
integrated  into  the  ACV  by  the  contractor  and  is  a 
version  of  a  similar  system  developed  for  the  Navy's 
Mark  VI  Patrol  Boat.  RWS  successfully  completed 
water  tank  drop  testing  in  June  2016. 

Design  Maturity 

According  to  the  program  office,  the  two  ACV 
contractors  released  90  percent  of  their  drawings  at 
the  time  of  the  combined  preliminary/critical  design 
review  in  October  2016.  Each  contractor  is 
expected  to  provide  a  vehicle  hull  that  will 
accommodate  a  minimum  crew  of  three  along  with 
10  combat  equipped  Marines.  Both  contractors  are 
now  developing  prototypes  of  their  designs,  each 
using  some  form  of  a  "V"  hull  design  for  added  force 
protection  from  underside  blasts  and  improved 
seats  to  protect  occupants.  The  Marine  Corps 
expects  prototype  delivery  ahead  of  the  contractual 
due  date  of  March  2017. 

ACV  Increment  1.1  units  will  rely  on  connector 
vessels  to  transport  them  from  amphibious  ships  to 
the  shore.  The  Marine  Corps  anticipates  ACV 
Increment  1 .2  will  be  capable  of  self-deploying  to 
shore.  According  to  program  officials,  the  ACV 
contractor  teams  designed  Increment  1.1  vehicles 
with  sufficient  growth  margins  to  accomodate  the 
requirements  of  Increment  1 .2.  The  Marine  Corps  is 
also  investing  in  studies  that  could  lead  to  a  high 
water  speed  vehicle  for  ACV  Increment  2.0. 

Production  Maturity 

The  ACV  program's  schedule  includes  concurrent 
testing  and  production  activities  that  could  leave  the 
program  with  little  flexibility  to  correct  deficiencies 
identified  during  developmental  testing  and  result  in 


costly  retrofits.  The  Marine  Corps  plans  to  select  a 
single  vendor  in  201 8  to  produce  the  ACV,  whereas 
system-level  developmental  testing  is  not  expected 
to  complete  until  December  2019,  approximately 
one  and  a  half  years  after  production  start.  The 
program  office  has  plans  for  a  corrective  action 
period  for  the  first  lot  and  potentially  second  lot  of 
low-rate  production  vehicles  to  implement  any 
remaining  modifications  resulting  from 
developmental  testing.  According  to  the  program 
office,  the  Marine  Corps  already  budgeted  for  these 
potential  retrofits.  Program  officials  reported  they 
plan  to  demonstrate  critical  manufacturing 
processes  on  a  pilot  production  line  prior  to  entering 
production  and  that  low-rate  production  quantities 
comprise  more  than  10  percent  of  the  total  ACV 
Increment  1.1  production  quantity,  but  attributed  this 
decision  to  ensuring  an  efficient  ramp  up  to  full-rate 
production. 

Other  Program  Issues 

Contract  performance  was  delayed  due  to  a  bid 
protest  filed  with  GAO,  which  was  denied  in  March 
2016.  According  to  the  program  office,  the  bid 
protest  pushed  the  expected  production  decision 
out  to  the  third  quarter  of  fiscal  year  201 8.  In  the 
fiscal  year  2017  budget,  the  Marine  Corps  funded 
ACV  Increment  1.1  to  the  level  identified  in  the 
program's  independent  cost  estimate. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  stated  that,  although  the  ACV 
program  leverages  previous  EFV  program  work,  it 
predominantly  utilized  the  Marine  Personnel  Carrier 
technology  demonstrator  and  prototype 
demonstration  efforts  to  inform  ACV  requirements. 
Program  officials  also  noted  program  funding  is 
depicted  in  base  year  fiscal  year  2017  dollars, 
making  costs  appear  greater  than  if  presented  in  the 
program's  base  year  fiscal  year  2014  dollars. 
Program  officials  further  noted  that  "Program  unit 
cost"  depicts  Program  Acquisition  Unit  Cost,  not 
Average  Procurement  Unit  Cost  (APUC).  The 
program  unit  cost  value  includes  military 
construction  funding  and  acquisition  operations  and 
maintenance  funding,  which  are  not  included  in 
APUC. 
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Lead  Component:  Navy 


Common  Name:  CH-53K 


CH-53K  Heavy  Lift  Replacement  Helicopter  (CH-53K) 


The  Marine  Corps'  CH-53K  heavy-lift  helicopter  is 
intended  to  transport  armored  vehicles,  equipment, 
and  personnel  to  support  operations  deep  inland 
from  a  sea-based  center  of  operations.  The  CH-53K 
is  expected  to  replace  the  legacy  CH-53E  helicopter 
and  provide  increased  range  and  payload, 
survivability  and  force  protection,  reliability  and 
maintainability,  and  coordination  with  other  assets, 
while  reducing  total  ownership  costs. 


Source:  Sikorsky  Aircraft  Corporation. 


System  development 


Program  Development  Design 

start  start  review 

<11/03)  (12/05)  (7/10) 


▲ 

Production 
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Production 


GAO  Low-rate 

review  decision 

(1/17)  (3/17) 


Operational  test  Initial 
complete  capability 
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Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  in  millions) 

Prime  contractor:  Sikorsky  Aircraft 

As  of 

Latest 

Percent 

Corporation 

12/2005 

08/2016 

change 

Program  office:  Patuxent  River,  MD 

Research  and  development  cost 

$4,749.5 

$6,936.7 

46.1% 

Funding  needed  to  complete: 

Procurement  cost 

$13,208.9 

$19,273.1 

45.9% 

R&D:  $1,130.2  million 

Total  program  cost 

$17,958.4 

$26,251.9 

46.2% 

Procurement:  $19,232.3  million 

Program  unit  cost 

$115,118 

$131,259 

14.0% 

Total  funding:  $20,388.0  million 

Total  quantities 

156 

200 

28.2% 

Procurement  quantity:  194 

Acquisition  cycle  time  (months) 

119 

168 

41.2% 

CH-53K  critical  technologies  are  approaching 
maturity  and  the  program  is  completing  flight  tests 
to  demonstrate  recent  redesigns  to  key 
components.  These  redesigns,  which  follow  the 
program's  July  2010  critical  design  review,  have 
delayed  aircraft  assembly  and  testing  and  slowed 
delivery  of  test  aircraft.  As  a  result,  the  program 
has  delayed  its  entry  into  low-rate  initial 
production  until  March  31, 2017.  The  program 
also  faces  uncertainty  associated  with  the 
relocation  of  the  CH-53K  production  facility,  and 
officials  report  they  continue  to  assess  potential 
impacts.  The  program  has  also  accepted  delivery 
of  its  four  test  aircraft  and,  as  of  January  2017, 
has  completed  over  370  flight  test  hours. 
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Attainment  of  Product  Knowledge 


As  of  January  2017 


Resources  and  requirements  match 


Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 


STATUS  CURRENT 
AT  STATUS 


DEVELOPMENT 

START 


O 


Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 


o  o 

o  • 


Complete  a  system-level  preliminary  design  review 


Product  design  is  stable 


Release  at  least  90  percent  of  design  drawings 
Test  a  system-level  integrated  prototype 


DESIGN  REVIEW 


o 


Manufacturing  processes  are  mature 


Demonstrate  Manufacturing  Readiness  Level  of  at  least 
9  or  critical  processes  are  in  statistical  control 

Demonstrate  critical  processes  on  a  pilot  production  line 
Test  a  production-representative  prototype 


PRODUCTION 

START 


^  Knowledge  attained 
o  Knowledge  not  attained 


Information  not  available 
Not  applicable 


GAO-1 7-333SP  Assessments  of  Selected  Weapon  Programs 


Common  Name:  CH-53K 


CH-53K  Program 

Technology  and  Design  Maturity 

The  CH-53K  program  entered  system  development 
in  December  2005  with  immature  critical 
technologies — an  approach  inconsistent  with  best 
practices.  These  technologies,  which  include  the 
main  rotor  blade  and  main  gear  box,  were 
approaching  maturity  at  the  program's  critical 
design  review  (CDR)  in  July  2010.  The  program 
began  flight  testing  both  of  the  technologies  in 
October  2015.  Although  this  testing  constitutes  an 
operational  environment,  the  technologies  have  not 
yet  been  assessed  for  final  form,  fit,  and  function. 

At  CDR,  the  program  assessed  what  was  later 
proven  to  be  an  unstable  design.  The  extent  of  this 
instability  is  unknown,  however,  as  the  program  no 
longer  tracks  information  on  design  drawings. 
Unanticipated  design  changes  to  non-critical 
technology  components  after  CDR  caused  test  and 
production  delays.  We  previously  found  that 
components  within  the  rear  module  assembly,  part 
of  the  main  gear  box,  required  a  number  of 
redesigns.  For  example,  we  found  equipment 
holding  one  of  the  gears  in  place  failed,  which 
required  a  reconfiguration  of  the  rear  module 
assembly  retention  design.  The  contractor 
completed  and  installed  the  new  design  on  each  of 
the  five  test  aircraft.  Program  officials  reported  that 
since  the  redesign,  the  program  has  successfully 
tested  the  main  gear  box,  and  actions  are  underway 
to  mitigate  the  remaining  design  risks  in  the 
program.  The  program  estimates  the  CH-53K  will 
not  only  meet,  but  will  exceed  all  of  its  key 
performance  requirements. 

Production  Maturity 

To  date,  the  CH-53K  program  has  taken  delivery  of 
five  test  vehicles,  which  includes  the  ground  test 
vehicle  and  four  flight  test  aircraft.  All  four  aircraft 
were  modified  to  include  the  redesigned  main  gear 
box  and  have  entered  flight  testing — accumulating 
over  370  flight  test  hours,  as  of  January  2017. 
However,  delays  producing  test  articles  have 
slowed  the  overall  schedule.  As  a  result,  the 
program  postponed  its  low-rate  initial  production 
decision  to  March  31 , 2017.  This  delay  exceeded 
the  program’s  schedule  parameters,  which  required 
the  Marine  Corps  to  report  the  delay  to  the  Under 
Secretary  of  Defense  for  Acquisition,  Technology 
and  Logistics  (USD  AT&L). 


USD  AT&L  subsequently  approved  a  schedule 
extension  and  decided  not  to  require  the  program  to 
establish  a  new  acquisition  program  baseline  until 
the  CH-53K's  low-rate  production  decision  is 
reached.  Our  best  practices  work  has  shown  that  if 
a  program's  critical  manufacturing  processes  are 
not  demonstrated  and  in  control  before  production 
begins,  it  is  at  risk  of  increased  cost  and  schedule. 
The  program  is  nearing  its  expected  low-rate 
production  decision  on  March  31, 2017,  and  does 
not  have  all  of  its  critical  manufacturing  processes 
under  statistical  control,  nor  has  its  manufacturing 
readiness  level  been  assessed  at  DOD's 
recommended  level  for  starting  production. 

According  to  program  officials,  production  will  be 
moved  from  its  current  location  in  West  Palm 
Beach,  Florida,  to  Sikorsky’s  headquarters,  which  is 
located  in  Stratford,  Connecticut.  According  to 
officials,  the  current  location  in  West  Palm  Beach  is 
owned  by  United  Technologies — Sikorsky’s  prior 
owner — and  Lockheed  Martin’s  acquisition  of 
Sikorsky  necessitated  the  change.  The  move  will 
require  a  number  of  equipment  and  configuration 
changes  to  Sikorsky’s  Stratford  facility,  which  will 
take  time  to  complete  and  pose  risk  to  the  CH-53K 
production  schedule.  Program  officials  continue  to 
assess  the  potential  impacts  of  the  production 
relocation. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  concurred  with  the  contents,  but 
provided  additional  information.  While  the  program's 
production  readiness  was  assessed  as  below 
DOD's  required  level  for  production,  the  program 
expects  to  meet  DOD's  recommended  standard 
upon  successful  completion  of  initial  operational  test 
and  evaluation.  In  addition,  the  program  office 
stated  the  movement  of  the  production  facility  was 
driven  by  required  capacity  and  not  the  Lockheed 
Martin  acquisition.  The  program  is  monitoring  the 
contractor's  move  plans,  which  just  competed  the 
second  of  six  gates.  There  is  a  planned  gap  of  8 
months  between  the  completion  of  the  fourth  and 
fifth  system  demonstration  test  articles  in  order  to 
facilitate  the  final  assembly  move.  The  program  also 
provided  other  technical  comments,  which  were 
incorporated  where  deemed  appropriate. 
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Lead  Component:  Navy 


Common  Name:  CVN  78 


CVN  78  Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier  (CVN  78) 

The  Navy  developed  the  Ford-class  nuclear- 
powered  aircraft  carrier  to  introduce  new  propulsion, 
aircraft  launch  and  recovery,  and  survivability 
capabilities  to  the  carrier  fleet.  The  Ford-class  is  the 
successor  to  the  Nimitz-class  aircraft  carrier,  and  its 
new  technologies  are  intended  to  create  operational 
efficiencies  while  enabling  a  25  percent  increase  in 
operational  aircraft  flights,  as  compared  to  legacy 
carriers.  The  Navy  also  expects  the  new 
technologies  to  enable  Ford-class  carriers  to 
operate  with  reduced  manpower. 


Source:  U.S.  Navy. 
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Program  Essentials 

Prime  contractor:  Huntington  Ingalls 
Industries 

Program  office:  Washington,  DC 
Funding  needed  to  complete: 

R&D:  $624.9  million 
Procurement:  $12,620.7  million 
Total  funding:  $13,275.2  million 
Procurement  quantity:  1 


Program  Performance  (fiscal  year  2017  dollars  in  millions) 


As  of 
04/2004 

Latest 

09/2016 

Percent 

change 

Research  and  development  cost 

$5,210.1 

$5,486.3 

5.3% 

Procurement  cost 

$33,377.1 

$31,918.1 

-4.4% 

Total  program  cost 

$38,587.3 

$37,539.7 

-2.7% 

Program  unit  cost 

$12,862,418 

$12,513,220 

-2.7% 

Total  quantities 

3 

3 

0.0% 

Acquisition  cycle  time  (months) 

137 

TBD 

NA 

Total  program  and  unit  cost  decreases  are  driven  by  a  decrease  in  the  estimated  costs  for  the  third 
ship  (CVN  80),  which  has  not  yet  been  validated  by  an  independent  cost  estimate. 

The  Navy  reported  9  of  the  program's  1 3  critical 
technologies  are  mature,  though  testing 
continues  to  reveal  issues  and  delay  lead  ship 
(CVN  78)  delivery — most  recently  due  to 
problems  with  the  ship's  propulsion  plant.  CVN  78 
construction  is  nearly  complete,  but  the  Navy 
reported  ship  delivery  was  delayed  to  April  2017. 
CVN  78  procurement  costs  remain  at  the  limit  of 
the  current  legislated  cost  cap  of  $12.9  billion, 
and  the  Navy  has  identified  $48  million  in  cost  risk 
from  extended  testing.  The  National  Defense 
Authorization  Act  for  Fiscal  Year  2016  reduced 
the  cost  cap  for  the  first  follow-on  ship  (CVN  79) 
to  $1 1 .4  billion,  though  we  found  the  funds  the 
Navy  has  budgeted  are  likely  insufficient  to 
complete  ship  construction.  The  Navy  adopted  a 
two-phased  acquisition  approach  for  CVN  79  and 
has  deferred  certain  construction  activities  and 
costs  to  after  ship  delivery. 
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Attainment  of  Product  Knowledge 


As  of  January  2017 

STATUS  CURRENT 
AT  STATUS 

Resources  and  requirements  match 

•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

DETAIL  DESIGN 
CONTRACT 
AWARD 

O 

• 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

O 

o 

•  Complete  a  system-level  preliminary  design  review 

O 

• 

Product  design  is  stable 

•  Complete  basic  and  functional  design  to  include 

100  percent  of  3D  product  modeling 

FABRICATION 

START 

O 

• 

^  Knowledge  attained  ■  ■  ■  ■  Information  not  available 
o  Knowledge  not  attained  Not  applicable 


Common  Name:  CVN  78 


CVN  78  Program 

Technology,  Design,  and  Production  Maturity 

The  Navy  reported  9  of  the  program’s  13  critical 
technologies  are  mature,  though  testing  continues 
to  reveal  issues.  CVN  78  began  construction  with 
immature  technologies  and  an  incomplete  design, 
leading  to  cost  and  schedule  growth.  The  Navy 
completed  deadload  testing  of  the  electromagnetic 
aircraft  launch  system  from  the  ship's  deck  in  2016. 
The  advanced  arresting  gear  (AAG)  began 
shipboard  testing  in  July  2015,  with  projected 
completion  in  March  2017.  The  dual  band  radar 
(DBR)  also  began  shipboard  testing  in  2015, 
despite  known  problems  that  could  affect  air  traffic 
control  functionality.  Both  the  AAG  and  DBR  are 
engaged  in  concurrent  land-based  testing.  The 
advanced  weapons  elevators  are  also  experiencing 
problems,  and,  as  of  January  2017,  the  Navy 
projected  that  only  2  of  the  11  elevators  would  be 
built  and  tested  by  ship  delivery.  In  June  2016,  a 
turbine  generator  in  the  ship's  propulsion  plant 
experienced  a  catastrophic  failure,  likely  due  to  a 
manufacturing  defect.  A  follow-on  review  revealed 
additional  problems  requiring  design  modifications 
to  the  voltage  regulator  and  protection  systems, 
which  has  delayed  the  propulsion  plant  testing 
schedule.  As  a  result  of  key  subsystem  deficiencies, 
the  Under  Secretary  of  Defense  for  Acquisition, 
Technology  and  Logistics  directed  an  independent 
review  team  to  identify  and  mitigate  potential 
technology  risks  for  follow-on  ships. 

The  Navy  recently  reported  additional  schedule 
delays  for  CVN  78  delivery  and  major  post-delivery 
test  events.  Construction  continues  on  CVN  79, 
which  is  23  percent  complete.  This  ship  uses  the 
CVN  78  design  with  some  modifications,  namely 
replacing  DBR  with  the  Enterprise  Air  Surveillance 
Radar  suite.  The  Navy  awarded  the  detail  design 
and  construction  contract  for  CVN  79  in  June  2015 
and  the  advance  procurement  contract  for  CVN  80 
in  May  2016. 

Other  Program  Issues 

In  2007,  Congress  established  a  procurement  cost 
cap  of  $10.5  billion  for  CVN  78,  and  lead  ship 
procurement  costs  have  since  increased  by  23 
percent  to  the  current  cost  cap  of  $12.9  billion. 
While  CVN  78  construction  is  nearly  complete,  the 
Navy  identified  $48  million  in  cost  risk  and  more 
recent  technical  issues  suggest  additional  costs, 


meaning  that  the  current  cost  cap  likely  does  not 
represent  necessary  funding  to  deliver  a  complete 
ship.  The  National  Defense  Authorization  Act  for 
Fiscal  Year  2016  reduced  the  cap  for  follow-on 
ships,  including  CVN  79’s  to  $11.4  billion,  though 
costs  for  this  ship  may  also  increase.  The  Office  of 
the  Secretary  of  Defense  projected  CVN  79  would 
exceed  the  program's  cost  estimate  by  about  $235 
million.  We  also  found  that  the  funds  the  Navy  has 
budgeted  based  on  CVN  79's  cost  estimate  are 
likely  to  be  insufficient  to  complete  ship 
construction.  The  Navy  has  expressed  confidence 
that  CVN  79  will  deliver  within  its  cost  cap,  which 
assumes  unprecedented  efficiency  gains  in 
construction — namely  that  CVN  79  production  hours 
will  be  significantly  lower  than  CVN  78.  The  Navy 
adopted  a  two-phase  acquisition  approach  for  CVN 
79  to  shift  some  construction  to  a  post-delivery 
period.  This  strategy  results  in  a  less  capable  and 
complete  ship  at  delivery,  and  transfers  the  costs  of 
known  capability  upgrades  to  other  accounts  by 
deferring  work  to  future  maintenance  periods — 
obscuring  CVN  79's  actual  costs. 

Program  Office  Comments 

With  93  percent  of  the  CVN  78's  test  program 
complete,  the  Navy  has  made  progress  resolving 
first-of-class  issues  and  has  resumed  critical-path 
testing  to  support  delivery  in  April  201 7.  The  Navy  is 
managing  the  $48  million  cost  risk  associated  with 
extended  shipboard  testing  and  the  costs  for 
schedule  delay  to  delivery.  The  Navy  continues  to 
pursue  cost  mitigation  within  the  program  to  offset 
this  cost  risk  to  deliver  the  ship  within  the  cost  cap. 
According  to  the  program,  with  26  percent  of  CVN 
79's  construction  complete,  cost  performance  to- 
date  remains  aligned  with  the  contract  target  to 
realize  an  18  percent  reduction  in  production  labor 
hours  from  CVN  78.  The  Navy  will  deliver  a 
complete  and  deployable  ship  at  the  end  of  Phase 
II,  with  the  construction  costs  from  both  phases 
accounted  within  the  cost  cap. 

GAO  Response 

Navy  cost  mitigation  plans  will  not  reduce  actual 
CVN  78  costs;  instead,  the  costs  will  be  shifted  to 
follow-on  ships  or  to  other  support  accounts.  CVN 
79  cost  performance  is  degrading,  and,  if  trends 
continue,  the  ship  is  at  risk  of  exceeding  its  cost 
cap. 
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Lead  Component:  Navy 


Common  Name:  DDG  1000 


DDG  1000  Zumwalt  Class  Destroyer  (DDG  1000) 

The  DDG  1000  destroyer  is  a  multi-mission  surface 
ship  designed  to  provide  advanced  capability  for 
littoral  operations  and  land  attack  to  support  forces 
ashore.  DDG  1 000  class  ships  feature  an  integrated 
power  system  and  a  total  ship  computing 
environment.  The  Navy  accepted  delivery  of  the 
lead  ship — comprised  of  the  hull,  mechanical,  and 
electrical  systems — in  May  2016.  The  Navy  has 
scheduled  a  separate,  post-delivery  phase  to 
activate  and  test  the  lead  ships  combat  systems. 

The  remaining  two  ships  of  the  class  are  under 
construction. 

Source:  U.S.  Navy. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  in  millions) 

Prime  contractors:  BAE  Systems, 

As  of 

Latest 

Percent 

General  Dynamics  Bath  Iron  Works, 

1/1998 

10/2016 

change 

Huntington  Ingalls,  and  Raytheon 

Research  and  development  cost 

$2,471.1 

$10,931.2 

342.4% 

Program  office:  Washington,  DC 

Procurement  cost 

$35,280.4 

$12,714.5 

-64.0% 

Funding  needed  to  complete: 

Total  program  cost 

$37,751.5 

$23,645.7 

-37.4% 

R&D:  $96.3  million 

Program  unit  cost 

$1,179,734 

$7,881,885 

568.1% 

Procurement:  $664.7  million 

Total  quantities 

32 

3 

-90.6% 

Total  funding:  $761 .0  million 

Acquisition  cycle  time  (months) 

128 

264 

1 06.3% 

Procurement  quantity:  0 

To  date,  the  Navy  has  fully  matured  fewer  than 
half  of  DDG  1000's  11  critical  technologies.  The 
program  reports  that  the  ship’s  design  is  stable, 
but  ongoing  development  and  shipboard  testing 
of  technologies  pose  risk  for  design  changes.  The 
201 6  delivery  of  the  lead  ship’s  hull,  mechanical, 
and  electrical  systems  followed  significant 
technical  challenges  with  the  integrated  power 
system,  which  contributed  to  an  18-month  delay. 
As  the  program  begins  to  test  the  lead  ship's 
combat  systems — an  effort  that  relies  on  stable 
power  from  the  integrated  power  system — it  faces 
challenges  achieving  its  cost  and  schedule 
baselines.  After  completing  delayed  combat 
system  acceptance  trials  for  the  lead  ship,  the 
Navy  will  conduct  initial  operational  testing.  The 
program’s  planned  date  for  initial  operating 
capability  has  slipped  more  than  2  years  from  July 
201 6  to  December  2019. 
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Attainment  of  Product  Knowledge 


As  of  January  2017 

STATUS 

AT 

CURRENT 

STATUS 

Resources  and  requirements  match 

•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

DETAIL  DESIGN 
CONTRACT 

AWARD 

O  • 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

O 

O 

•  Complete  a  system-level  preliminary  design  review 

• 

• 

Product  design  is  stable 

•  Complete  basic  and  functional  design  to  include 

100  percent  of  3D  product  modeling 

DESIGN 

START 

O 

• 

^  Knowledge  attained 
O  Knowledge  not  attained 


Information  not  available 
Not  applicable 


Common  Name:  DDG  1000 


DDG  1000  Program 

Technology  Maturity 

At  start  of  detail  design  in  2005,  the  DDG  1000 
program  had  matured  1  of  its  current  11  critical 
technologies — an  acquisition  approach  inconsistent 
with  best  practices.  The  DDG  1000  program  has 
since  fully  matured  5  of  11  critical  technologies.  The 
program  states  that  5  of  the  remaining  6  will  be 
demonstrated  during  post-delivery  availability  and 
combat  systems  activation,  extending  from  the 
second  quarter  of  fiscal  year  201 7  to  the  first  quarter 
of  fiscal  year  2019.  The  Navy  has  since  delayed  the 
start  of  this  activity  to  early  201 8.  Prior  to  the  May 
201 6  delivery  of  the  lead  ship's  hull,  mechanical, 
and  electrical  systems,  the  program  experienced 
significant  technical  issues  with  the  integrated 
power  system,  a  critical  technology  which  supplies 
power  to  the  ship's  propulsion  and  electronic 
systems.  Challenges  were  due,  in  part,  to  the 
Navy's  decision  not  to  fully  test  and  validate  the 
performance  of  the  system  in  a  representative 
environment  prior  to  installation  on  the  ship. 
Program  officials  noted  that  combat  systems  testing 
and  activation  relies  on  stable  power,  and  will 
introduce  new  challenges  for  the  power  system 
beyond  those  encountered  to  date.  After  scheduling 
combat  systems  acceptance  trials  for  the  lead  ship 
in  the  third  quarter  of  fiscal  year  201 7,  the  Navy  has 
delayed  this  activity  to  early  2018. 

The  program  reported  land-based  testing  of 
modifications  to  the  multifunction  radar,  to  include  a 
volume  search  capability,  is  complete.  In  2017, 
testing  of  the  modified  multifunction  radar  will  move 
to  a  Navy  test  bed  for  ship  self-defense,  before 
initial  operational  testing  aboard  the  lead  ship.  The 
program  also  reported  that  the  planned  date  for 
completion  of  software  development  for  the  class 
was  delayed  from  January  2016  to  December  2017 
to  prioritize  cybersecurity  enhancements  and 
software  corrections  related  to  integration  of  the 
ship's  power  and  engineering  control  systems.  The 
program  did  not  make  a  low-rate  initial  production 
decision  on  the  long-range  land-attack  projectile  in 
fiscal  year  2016  as  planned. 

Design  and  Production  Maturity 

The  DDG  1000  design  was  not  stable  at  lead  ship 
fabrication  start  in  2009.  Since  then,  the  Navy  and 
its  contractors  stabilized  the  design,  but  ongoing 
development  and  shipboard  testing  of  technologies 


may  result  in  design  changes.  Delivery  of  the  lead 
ship's  hull,  mechanical,  and  electrical  systems  was 
18  months  behind  schedule  due  in  part  to 
challenges  completing  electrical  work,  with  the 
shipbuilder  citing  resource  shortages  and  workforce 
turnover.  Program  officials  noted  the  lead  ship  will 
not  complete  final  contract  trials,  foregoing  an 
opportunity  to  identify  and  mitigate  technical  and 
design  deficiencies  prior  to  completing  construction 
of  the  remaining  two  ships. 

As  of  October  2016,  construction  of  the  two 
remaining  ships  was  91  and  59  percent  complete, 
respectively.  Program  officials  noted  the  shipbuilder 
continues  to  face  challenges  in  completing  electrical 
work  and  since  March  2016,  delivery  dates  for  the 
remaining  two  ships  have  each  slipped  by  about  two 
fiscal  quarters.  With  the  Navy  as  lead  integrator, 
program  officials  noted  that  timely  delivery  of 
government-furnished  equipment  to  the  shipbuilder 
will  be  critical  to  achieving  cost  and  schedule 
baselines  for  these  ships'  hulls,  under  the  terms  of 
their  fixed-price  construction  contracts. 

Other  Program  Issues 

During  the  lead  ship's  transit  between  its  Maine 
construction  site  and  California  home  port,  the 
ship's  propulsion  system  experienced  two  seawater 
intrusions  which  required  unscheduled  repairs. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment, 
program  officials  stated  the  program  reached  a 
significant  milestone  with  delivery  of  the  first-of- 
class  ship  in  fiscal  year  2016.  Program  officials 
noted  that  DDG  1000  underwent  an  extensive 
period  of  testing  including  three  sets  of  trials  prior  to 
delivery  and  the  ship  continued  test  and  activation 
activities  during  its  transit  to  San  Diego.  Officials 
also  noted  that  during  transit,  seawater 
contamination  occurred  in  two  propulsion  motor 
bearing  lubricating  oil  sumps  and  it  is  not 
uncommon  for  first-of-class  ships  to  identify 
deficiencies  and  undergo  repairs  during  underway 
periods  following  construction.  According  to 
program  officials,  the  ship’s  post-delivery  availability 
will  include  periods  of  in-port  and  at-sea  testing  and 
activation  of  ship  systems.  Finally  the  program 
noted  that  following  combat  system  activation,  the 
ship  will  conduct  dockside  and  at-sea  trials  as  well 
as  start  operational  testing. 
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Lead  Component:  Navy 


Common  Name:  G/ATOR 


Ground/Air  Task  Oriented  Radar  (G/ATOR) 

The  Marine  Corps'  Ground/Air  Task  Oriented  Radar 
(G/ATOR)  is  a  three-dimensional,  short-to-medium 
range,  multi-role  radar  designed  to  detect,  identify, 
and  track  threats  such  as  incoming  cruise  missiles, 
rockets,  and  artillery.  It  will  replace  five  legacy 
radars.  G/ATOR  is  being  acquired  in  blocks,  with 
later  blocks  focused  on  software  upgrades.  We 
assessed  Block  1 ,  which  has  an  air  defense  and 
surveillance  role,  and  made  observations  on  Block 
2,  which  will  determine  enemy  firing  positions  and 
point  of  impact  for  incoming  fire. 


Source:  U.S.  Marine  Corps. 
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(3/09) 
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(1/17) 

(6/17) 
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(3/19) 

Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  in  millions) 

Prime  contractor:  Northrop  Grumman 

As  of 

Latest 

Percent 

Program  office:  Quantico,  VA 

08/2005 

12/2015 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$384.1 

$1,061.9 

1 76.5% 

R&D:  $185.1  million 

Procurement  cost 

$1,206.0 

$1,752.8 

45.3% 

Procurement:  $1 ,346.5  million 

Total  program  cost 

$1,590.1 

$2,818.4 

77.3% 

Total  funding:  $1 ,531 .6  million 

Program  unit  cost 

$24,845 

$62,632 

152.1% 

Procurement  quantity:  36 

Total  quantities 

64 

45 

-27.7% 

Acquisition  cycle  time  (months) 

66 

155 

134.8% 

The  cost  data  includes  G/ATOR  Blocks  1  and  2 

The  acquisition  cycle  time  was  calculated  for  Block  1 

only. 

The  G/ATOR  program  received  approval  to  enter 
low-rate  initial  production  (LRIP)  in  March  2014 
with  mature  technologies,  a  stable  design,  and 
production  processes  that  were  demonstrated, 
but  not  in  control.  In  early  testing,  software 
stability  was  a  major  problem  that  degraded 
reliability.  Since  then,  software  performance  and 
reliability  has  improved  and  the  program 
continues  to  make  software  updates  to  address 
quality  and  reliability  issues.  The  next  opportunity 
to  collect  representative  data  on  reliability  will  be 
in  2017.  Program  officials  noted  G/ATOR 
transitioned  to  a  new  semiconductor  technology 
in  2016,  which  will  be  used  in  initial  operational 
testing  and  for  all  future  radars  produced  for  the 
program.  A  revised  test  and  evaluation  master 
plan  (TEMP)  capturing  the  new  technology  was 
approved  by  Director,  Operational  Test  and 
Evaluation  (DOT&E)  in  March  2016. 
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Attainment  of  Product  Knowledge 


As  of  January  2017 


Resources  and  requirements  match 


STATUS  CURRENT 
AT  STATUS 


Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

Complete  a  system-level  preliminary  design  review 


DEVELOPMENT 

START 


o 


Product  design  is  stable 


Release  at  least  90  percent  of  design  drawings 
Test  a  system-level  integrated  prototype 


DESIGN  REVIEW 


o 

o 


Manufacturing  processes  are  mature 


Demonstrate  Manufacturing  Readiness  Level  of  at  least 
9  or  critical  processes  are  in  statistical  control 


PRODUCTION 

START 


o  o 


Demonstrate  critical  processes  on  a  pilot  production  line 
Test  a  production-representative  prototype 


^  Knowledge  attained 
o  Knowledge  not  attained 


Information  not  available 
Not  applicable 


Common  Name:  G/ATOR 


G/ATOR  Program 

Technology  and  Design  Maturity 

As  of  August  201 6,  the  program  reported  that  all  six 
G/ATOR  critical  technologies  are  fully  mature. 
Further,  the  program  reported  a  mature  design,  with 
more  than  99  percent  of  design  drawings  released. 
Although  G/ATOR  hardware  has  proven  to  be 
reliable,  software  maturity  and  quality  problems 
have  affected  the  reliability  of  the  overall  system. 
After  G/ATOR  failed  to  meet  its  reliability  target 
during  developmental  testing,  a  panel  of  experts 
convened  by  the  Navy  in  June  2014  conducted  an 
in-depth  analysis  of  the  program  and  found 
G/ATOR's  reliability  requirement  was  likely 
unachievable  and  did  not  reflect  operational  needs. 
In  March  2015,  the  Marine  Corps  clarified  the 
reliability  requirement  to  ensure  it  was  consistent 
with  G/ATOR's  operational  mission  profile.  Since 
then,  software  performance  and  reliability  has 
improved.  The  program  manager  noted  that  the 
program  continues  to  make  software  updates  to 
address  quality  and  reliability  issues,  and  that  the 
next  opportunity  to  collect  representative  reliability 
data  will  be  2017  when  the  first  LRIP  radar  is 
delivered  and  tested. 

Beginning  with  radars  produced  in  2016,  the 
program  upgraded  the  radar's  transmit  and  receive 
modules  by  using  a  newer  semiconductor 
technology.  According  to  program  officials,  this  new 
gallium  nitride  (GaN)  technology  is  mature  and  the 
new  modules  will  fit  inside  the  system  the  same  as 
the  older  gallium  arsenide  (GaAs)  modules,  which 
minimizes  design  changes.  The  GaN  technology  is 
expected  to  achieve  better  performance  with  higher 
reliability  at  a  lower  cost  by  reducing  the  number  of 
modules  required. 

Production  Maturity 

The  Assistant  Secretary  of  the  Navy  for  Research, 
Development  and  Acquisition  [ASN(RDA)] 
approved  the  program's  entry  into  LRIP  in  March 
201 4.  At  the  time  of  this  decision,  the  G/ATOR 
program  had  demonstrated  its  production 
processes  to  the  level  recommended  by  DOD,  but 
had  not  brought  them  into  statistical  control,  which 
is  inconsistent  with  best  practices.  This  status  is 
unchanged. 


In  August  2016,  ASN(RDA)  approved  an  increase  in 
LRIP  units  from  14  to  15  radar  systems.  According 
to  the  program  office,  the  first  6  of  the  15  units  will 
use  the  older  GaAs  modules  and  the  follow-on  9  will 
use  the  newer  GaN  modules.  The  program  office 
awarded  a  contract  in  August  2016  to  Northrop 
Grumman  to  procure  the  9  GaN  radars.  According 
to  the  program  office,  the  first  LRIP  deliveries  of 
GaAs  radars  and  GaN  radars  are  expected  in  early 
2017  and  2018,  respectively. 

Other  Program  Issues 

The  G/ATOR  program  revised  its  test  strategy  to 
address  reliability  concerns  raised  by  the  expert 
panel,  to  include  a  test  strategy  for  GaN,  and  clarify 
the  operational  reliability  requirements.  Originally, 
the  program  planned  to  conduct  initial  operational 
testing  with  the  older  GaAs  configuration,  but  DOD's 
DOT&E  raised  concerns  about  the  production 
representativeness  of  this  configuration,  as  a 
majority  of  the  planned  G/ATOR  procurements  are 
with  the  newer  GaN  modules.  Program  officials  said 
testing  using  GaN  will  occur  in  2018.  In  March  201 6, 
DOT&E  approved  a  revised  TEMP  capturing  the 
clarified  reliability  requirements  and  use  of  GaN 
technology. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  the  following  information. 
G/ATOR  has  remained  on  schedule  and  decreased 
the  estimated  total  program  cost  by  10  percent 
since  the  program  was  rebaselined  in  2010.  The 
program  expects  on-time  delivery  of  the  first  system 
in  February  2017,  demonstrating  the  program 
remains  on  schedule  and  production  processes  are 
in  control.  G/ATOR  has  successfully  demonstrated 
all  Block  1  key  performance  requirements  and  the 
capability  to  meet  all  Block  2  performance 
requirements.  There  are  15  LRIP  systems  under 
contract;  6  systems  will  be  delivered  in  2017  to 
support  developmental  testing  and  operational 
assessments  of  Blocks  1  and  2,  culminating  with 
initial  operational  capability  in  2018.  Three  systems 
will  be  delivered  in  2018  to  support  initial 
operational  test  and  evaluation  and  the  full  rate 
production  decision  in  2019.  G/ATOR's  operational 
availability  has  exceeded  objective  requirements 
and  early  software  quality  challenges  are  being 
addressed.  Program  officials  stated  they  are 
positioned  to  deliver  the  required  capabilities  to  the 
warfighter  per  their  commitments. 
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Lead  Component:  Navy 


Common  Name:  JPALS 


Joint  Precision  Approach  and  Landing  System  (JPALS) 

JPALS  is  a  program  to  develop  a  GPS-based 
landing  system  for  aircraft  carriers  and  amphibious 
assault  ships  to  support  operations  with  the  F-35 
Lightning  II  and  the  MQ-25  Unmanned  Aircraft 
System.  The  program  intends  to  provide  a  reliable 
sea-based  precision  approach  and  landing 
capability  in  adverse  weather  conditions.  The 
program  has  restructured  from  multiple  increments 
to  one,  and  incorporates  capabilities  into  its  base 
design  originally  planned  for  additional  increments. 


Source:  U.  S.  Navy. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017  dollars  in  millions) 

Prime  contractor:  Raytheon 

As  of 

Latest 

Percent 

Program  office:  Lexington  Park,  MD 

07/2008 

10/2016 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$859.2 

$1,449.8 

68.7% 

R&D:  $437.4  million 

Procurement  cost 

$231 .3 

$402.9 

74.2% 

Procurement:  $402.9  million 

Total  program  cost 

$1,098.0 

$1,860.2 

69.4% 

Total  funding:  $840.2  million 

Program  unit  cost 

$29,677 

$56,370 

89.9% 

Procurement  quantity:  23 

Total  quantities 

37 

33 

-10.8% 

Acquisition  cycle  time  (months) 

75 

194 

158.7% 

JPALS  originally  began  development  in  July 
2008,  but  the  Navy  restructured  the  program  in 
2014  due  to  affordability  concerns,  as  well  as 
other  military  services  and  civilian  plans  to 
continue  using  their  current  landing  systems.  The 
Navy  assessed  the  restructured  program  through 
a  new  system-level  preliminary  design  review  in 
March  2016  and  approved  a  development  restart 
in  June  2016.  The  program  previously  accepted 
delivery  of  eight  prototypes,  and  plans  to  procure 
two  additional  ones.  These  prototypes  are 
expected  to  support  JPALS  developmental  testing 
and  F-35  Lightning  II  initial  capability.  Program 
officials  reported  conducting  shipboard 
developmental  testing  in  2016,  and  the  program 
plans  to  hold  a  critical  design  review  in  March 
201 7.  The  JPALS  program  has  also  contracted  for 
a  trade  study  to  assess  options  for  integrating 
future  capability  updates. 


Attainment  of  Product  Knowledge 


As  of  January  2017 


Resources  and  requirements  match 


STATUS  CURRENT 
AT  STATUS 


•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

DEVELOPMENT 

START 

• 

• 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

o 

o 

•  Complete  a  system-level  preliminary  design  review 

o 

• 

Product  design  is  stable 

•  Release  at  least  90  percent  of  design  drawings 

DESIGN  REVIEW 

O 

• 

•  Test  a  system-level  integrated  prototype 

O 

• 

Manufacturing  processes  are  mature 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

PRODUCTION 

START 

9  or  critical  processes  are  in  statistical  control 
Demonstrate  critical  processes  on  a  pilot  production  line 
Test  a  production-representative  prototype 


^  Knowledge  attained 
O  Knowledge  not  attained 


Information  not  available 
Not  applicable 
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Common  Name:  JPALS 


JPALS  Program 

Technology,  Design,  and  Production  Maturity 

JPALS  entered  system  development  in  July  2008 
with  technologies  that  were  nearing  maturity.  The 
program  also  held  a  critical  design  review  in 
December  201 0,  which  evaluated  a  design  that  later 
proved  unstable.  In  June  2014,  the  Navy 
restructured  the  JPALS  program  due  to  funding 
concerns  and  to  accelerate  the  development  of 
aircraft  auto-land  capabilities.  The  Navy  also 
attributed  the  restructuring  to  a  review  it  conducted 
of  its  precision  approach  and  landing  capabilities,  as 
well  as  the  other  military  services  and  civilian  plans 
to  continue  use  of  their  current  landing  systems 
rather  than  investing  in  the  new  JPALS  capability. 

Following  the  program  restructuring,  the  Navy 
began  reassessing  JPALS's  technology  maturity 
and  design  stability  to  account  for  the  changes.  The 
program  reported  conducting  a  system 
requirements  review  in  March  2015  and  a  system- 
level  preliminary  design  review  (PDR)  in  March 
2016.  Subsequently,  the  Under  Secretary  of 
Defense  for  Acquisition,  Technology  and  Logistics 
delegated  milestone  decision  authority  to  the  Navy. 
In  June  2016,  the  Navy  approved  the  start  of 
engineering  and  manufacturing  development  for  the 
restructured  JPALS  program.  At  that  time,  the  Navy 
also  approved  the  program's  new  procurement  cost 
and  quantity  estimates.  Because  JPALS 
functionality  is  primarily  software-based,  the 
restructured  program  will  require  new  software 
development.  The  PDR  set  a  new  technology 
baseline  for  software  development  and  did  not 
identify  any  critical  manufacturing  processes.  The 
program's  next  technical  review  will  be  the  critical 
design  review  planned  for  March  2017.  At  that 
review,  the  program  will  update  the  total  number  of 
expected  design  drawings,  as  well  as  reevaluate  the 
need  for  any  critical  manufacturing  processes. 
Program  officials  reported  completing  shipboard 
developmental  testing  aboard  a  Nimitz-class  aircraft 
carrier  in  August  2016  and  beginning  testing  aboard 
an  America-class  amphibious  assault  ship  in 
October  2016. 

Prior  to  the  restructuring,  the  program  had  accepted 
delivery  of  eight  prototypes — also  referred  to  as 
engineering  development  models.  The  program 
awarded  a  contract  in  September  2016  to  upgrade 
these  prototypes,  as  well  as  to  procure  two 


additional  prototypes  for  developmental  testing. 
These  prototypes  will  be  production-representative, 
and  the  JPALS  program  plans  to  use  them  to 
provide  early  operational  capability  in  support  of  F- 
35  Lightning  II  initial  capability  in  fiscal  year  2018. 

Other  Program  Issues 

Because  JPALS  is  GPS-based,  it  will  need  to  be 
compliant  with  any  updates  to  DOD's  GPS  systems. 
The  JPALS  program  contracted  for  a  trade  study  to 
determine  future  GPS  integration  and 
implementation  options,  as  well  as  potential  costs 
and  schedule  issues.  The  program  estimates  the 
trade  study  will  be  completed  no  later  than  fiscal 
year  2019. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  noted  "fact-of-life"  changes  in  2013 
demanded  the  Navy's  Precision  Approach  and 
Landing  Capabilities  Roadmap  be  revisited.  By 
June  201 3,  the  Navy  validated  the  need  for  only  the 
ship-based  JPALS  requirements.  The  associated 
reduction  of  JPALS  quantities  precipitated  unit  cost 
growth  in  excess  of  statutory  critical  thresholds.  The 
Navy  subsequently  accelerated  future  JPALS 
capabilities  into  a  single  program  to  meet  the  sea- 
based  requirement.  The  program  reported 
completing  PDR  and  development  restart  in  March 
and  June  2016,  respectively. 

Military  GPS  user  equipment  purchased  after  fiscal 
year  2017  is  required  to  be  M  Code  capable — a  new 
military  GPS  signal  designed  to  further  improve  anti¬ 
jamming  and  secure  access  to  GPS  signals  for 
military  users.  JPALS  has  contracted  for  a  trade 
study  to  identify  alternatives  for  the  incorporation  of 
M  Code  into  the  JPALS  ship  systems.  The  results 
will  be  presented  no  later  than  fiscal  year  2019.  The 
program  office  also  provided  technical  comments, 
which  we  incorporated  where  appropriate. 
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Lead  Component:  Navy 


Common  Name:  LHA  6 


LHA  6  America  Class  Amphibious  Assault  Ship  (LHA  6) 

The  Navy’s  LHA  6  class  will  replace  the  LHA  1 
Tarawa-class  amphibious  assault  ships.  The  LHA  6 
class  design  is  based  on  the  fielded  LHD  8  and 
consists  of  three  ships.  The  ships  feature  enhanced 
aviation  capabilities  and  are  designed  to  support 
Marine  Corps  assets  in  an  expeditionary  strike 
group.  LHA  6  construction  began  in  December  2008 
with  delivery  in  April  2014.  LHA  7  construction 
began  in  July  2013  with  delivery  expected  in 
December  2018.  The  Navy  plans  to  exercise  an 
option  for  detail  design  and  construction  of  LHA  8  in 
summer  2017. 


Source:  Huntington  Ingalls  Industries 
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Program  Essentials 

Program  Performance  (fiscal  year  2017  dollars  in  millions) 

Prime  contractor:  Huntington  Ingalls 

As  of 

Latest 

Percent 

Incorporated 

01/2006 

09/2016 

change 

Program  office:  Washington,  DC 

Research  and  development  cost 

$239.6 

$454.2 

89.5% 

Funding  needed  to  complete: 

Procurement  cost 

$3,209.7 

$9,532.5 

197.0% 

R&D:  $38.6  million 

Total  program  cost 

$3,449.3 

$9,989.3 

189.6% 

Procurement:  $2,755.1  million 

Program  unit  cost 

$3,449,293 

$3,329,777 

-3.5% 

Total  funding:  $2,794.9  million 

Total  quantities 

1 

3 

200.0% 

Procurement  quantity:  1 

Acquisition  cycle  time  (months) 

146 

176 

20.5% 

LHA  6  lead  ship  construction  began  in  December 
2008  with  mature  technologies  but  an  incomplete 
design.  The  Navy  accepted  delivery  of  the  ship  in 
April  2014  after  a  20-month  delay,  and  the  ship 
has  begun  post-delivery  activities.  In  March  2016, 
the  program  completed  post  shakedown 
availability  and  achieved  initial  operating 
capability — 7  months  ahead  of  schedule.  LHA  7, 
which  largely  shares  the  LHA  6  design,  began 
construction  in  July  2013  and  is  expected  to 
deliver  in  December  2018.  Design  changes  to 
LHA  8  are  more  significant  and  include  the 
addition  of  a  well  deck.  The  program  awarded  an 
advanced  engineering  and  procurement  of  long 
lead  time  material  contract  for  LHA  8  in  June 
2016.  This  contract  includes  an  option  for  LHA  8 
detail  design  and  construction,  which  the  program 
intends  to  exercise  in  summer  2017. 


Attainment  of  Product  Knowledge 


As  of  January  2017 

STATUS  CURRENT  1 
AT  STATUS  1 

1  Resources  and  requirements  match 

•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

DETAIL  DESIGN 
CONTRACT 
AWARD 

• 

• 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

• 

• 

•  Complete  a  system-level  preliminary  design  review 

• 

• 

1  Product  design  is  stable 

•  Complete  basic  and  functional  design  to  include 

100  percent  of  3D  product  modeling 

FABRICATION 

START 

O 

• 

^  Knowledge  attained 
O  Knowledge  not  attained 


Information  not  available 
Not  applicable 
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Common  Name:  LHA  6 


LHA  6  Program 

Technology,  Design,  and  Production  Maturity 

The  program  awarded  a  construction  contract  for 
the  lead  ship,  LHA  6,  in  June  2007  after  determining 
the  ship  relied  on  six  non-critical  subsystems,  but  no 
critical  technologies,  to  achieve  capabilities.  The 
Navy  later  restructured  the  acquisition  program  for 
one  of  these  subsystems,  the  Joint  Precision 
Approach  and  Landing  System,  to  focus  it  on 
aircraft  auto-land  capabilities.  Full  integration  of  this 
subsystem  is  not  expected  until  2020.  In  the  interim, 
the  LHA  6  program  will  use  backup  aviation  control 
systems  to  meet  requirements. 

LHA  6  is  a  modification  of  the  LHD  8.  When  LHA  6 
construction  began  in  December  2008,  the  ship's 
design  was  only  77  percent  complete.  The  Navy 
accepted  delivery  of  LHA  6  in  April  2014,  20  months 
late  and  more  than  $500  million  over  cost.  The 
program  declared  initial  capability  in  March  2016. 
Officials  expect  operational  testing  of  LHA  6  to 
conclude  in  June  2017.  Following  delivery,  the 
program  had  to  execute  design  changes  to 
accommodate  the  F-35  at  a  cost  of  nearly  $60 
million.  According  to  officials,  testing  of  these 
modifications  to  accommodate  the  F-35  will 
complete  in  fiscal  year  2019.  LHA  6  is  scheduled  to 
conduct  its  first  deployment  in  mid-2017. 

Construction  of  the  first  follow-on  ship,  LHA  7, 
began  in  July  2013.  The  LHA  7  design  incorporates 
modifications  to  the  LHA  6  design,  and,  as  of 
September  2016,  approximately  58  percent  of  the 
ship  was  built.  The  program  plans  to  launch,  or 
transfer  to  water,  LHA  7  in  mid-2017.  The  program 
anticipates  a  delay  in  the  ship's  delivery  of 
approximately  6  months  to  December  2018  in  order 
to  execute  design  changes  required  for  operations 
with  the  F-35.  Program  officials  said  the  shipbuilder 
improved  its  performance  on  LHA  7  by 
implementing  lessons  learned  and  re-hiring  staff 
from  the  lead  ship  construction.  In  addition,  the 
Navy  provided  new  incentives  to  the  shipbuilder  to 
promote  a  higher  quality  of  work,  and  program 
officials  noted  improvements  since  these  were 
implemented.  However,  the  shipbuilder  is  not 
currently  performing  efficiently  enough  to  construct 
the  ship  within  planned  costs. 


The  Navy  held  a  limited  competition  between  two 
shipbuilders  and,  in  June  2016,  awarded  a  contract 
to  the  current  builder,  Huntington  Ingalls,  to  begin 
planning  LHA  8  detail  design  and  construction. 
Design  changes  on  LHA  8  will  be  more  significant 
than  those  on  LHA  7,  as  the  Navy  is  incorporating  a 
well  deck  to  accommodate  two  landing  craft.  To 
integrate  a  well  deck,  program  officials  estimate 
approximately  35  percent  of  LHA  8’s  design  will 
change.  Program  officials  told  us  they  expect  detail 
design  plans  to  include  three-dimensional  models, 
unlike  the  two-dimensional  drawings  used  for  the 
detail  design  of  LHA  6  and  LHA  7. 

Other  Program  Issues 

In  January  2015,  DOD's  Director,  Operational 
Testing  and  Evaluation  (DOT&E)  expressed 
concern  that  LHA  6's  self-defense  system  is 
incapable  of  defending  against  certain  anti-ship 
cruise  missile  threats.  In  January  2016,  DOT&E 
acknowledged  actions  taken  by  the  program  office 
to  improve  performance,  but  stated  the  LHA's  6’s 
self-defense  system  is  unlikely  to  meet  all 
requirements.  DOT&E  also  expressed  concerns 
regarding  the  performance  of  LHA  6's  missile 
launch  system.  In  response,  the  Navy  is 
investigating  options  to  modify  or  replace  the 
system,  but  in  the  interim  has  modified  launch 
procedures  and  increased  the  supply  of 
replacement  parts. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
Navy  reported  LHA  6  completed  its  post  shakedown 
availability  in  March  2016.  LHA  6's  initial  capability 
was  declared  less  than  one  week  later.  Program 
officials  reported  the  ship  is  currently  conducting 
fleet  exercises  and  other  required  certifications,  and 
they  expect  to  complete  LHA  6  Initial  Operational 
Test  and  Evaluation  in  summer  2017,  prior  to  initial 
deployment.  Program  officials  reported  LHA  7 
construction  is  not  experiencing  labor  resource 
issues  as  seen  in  the  past  and  continues  to  address 
labor  efficiency  concerns  to  control  cost.  Program 
officials  reported  LHA  7  is  on  schedule  to  meet  its 
contract  delivery  date  of  December  4,  201 8.  LHA  8’s 
planning,  advanced  engineering,  and  procurement 
of  long  lead  time  material  began  in  June  2016  with 
contract  award  to  Huntington  Ingalls.  Program 
officials  expect  to  exercise  the  LHA  8  detail  design 
and  construction  contract  option  in  summer  2017. 


Page  104 


GAO-1 7-333SP  Assessments  of  Selected  Weapon  Programs 


Lead  Component:  Navy 


Common  Name:  LCS 


Littoral  Combat  Ship  (LCS) 

The  Navy's  Littoral  Combat  Ship  (LCS)  is  designed 
to  perform  mine  countermeasures,  antisubmarine 
warfare,  and  surface  warfare  missions.  It  consists  of 
the  ship  itself,  or  seaframe,  and  the  mission 
package  it  deploys.  The  Navy  is  buying  two 
designs — the  Freedom  variant,  a  steel  monohull 
(LCS  1  and  odd  numbered  ships  built  by  Lockheed 
Martin),  and  the  Independence  variant,  an 
aluminum  trimaran  (LCS  2  and  even  numbered 
ships  built  by  Austal) — and  has  awarded  contracts 
for  26  seaframes  with  plans  for  2  more  in  fiscal  year 
2017.  We  assessed  both  designs. 

Sources:  Lockheed  Martin  (left);  General  Dynamics  (right). 
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Program  Essentials 

Prime  contractors:  Austal  USA, 
Lockheed  Martin 

Program  office:  Washington  Navy  Yard, 
DC 

Funding  needed  to  complete: 

R&D:  $678.9  million 
Procurement:  $1 0,291 .8  million 
Total  funding:  $11,082.1  million 
Procurement  quantity:  14 


Program  Performance  (fiscal  year  2017  dollars  in  millions) 


As  of 

Latest 

Percent 

05/2004 

10/2016 

change 

Research  and  development  cost 

$962.0 

$3,971.6 

312.8% 

Procurement  cost 

$511.6 

$22,429.2 

4,284.5% 

Total  program  cost 

$1,473.7 

$26,650.5 

1 ,078.4% 

Program  unit  cost 

$368,423 

$666,262 

80.8% 

Total  quantities 

4 

40 

900% 

Acquisition  cycle  time  (months) 

41 

119 

190.2% 

Cost  data  are  for  the  seaframe  program  only,  which  includes  the  cost  of  29  LCS  and  11  modified 
LCS/frigates  in  the  current  program  baseline.  Research  and  development  funding  includes  detail 
design  and  construction  of  two  ships. 


The  LCS  seaframe  program,  now  more  than  12 
years  into  production,  has  yet  to  demonstrate  the 
maturity  of  two  critical  technologies — the 
aluminum  hull  structure  and  launch,  handling,  and 
recovery  system  of  the  Independence  variant. 
The  Navy  undertook  shock  trials  on  both  variants 
as  well  as  a  survivability  trial  on  LCS  4.  Both  sets 
of  tests  revealed  design  deficiencies.  Since 
December  2015,  five  LCS  seaframes 
experienced  engineering  casualties  that  limited 
their  availability.  The  Navy  announced  significant 
changes  to  LCS  crewing  and  support  concepts  in 
response.  Both  shipbuilders  continue  to  deliver 
LCS  seaframes  significantly  behind  schedule  and 
in  excess  of  cost  targets. 


Attainment  of  Product  Knowledge 


As  of  January  2017 

STATUS  CURRENT 
AT  STATUS 

Resources  and  requirements  match 

•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

DETAIL  DESIGN 
CONTRACT 
AWARD 

O 

• 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

O 

o 

•  Complete  a  system-level  preliminary  design  review 

O 

• 

Product  design  is  stable 

•  Complete  basic  and  functional  design  to  include 

100  percent  of  3D  product  modeling 

FABRICATION 

START 

O 

• 

^  Knowledge  attained 
O  Knowledge  not  attained 


Information  not  available 
Not  applicable 
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Common  Name:  LCS 


LCS  Program 

Technology  Maturity 

Sixteen  of  the  18  critical  technologies — the  total 
number  of  technologies  for  both  designs — are 
mature.  However,  efforts  continue  to  further  mature 
two  Independence  variant  technologies — the 
aluminum  hull  structure  and  the  launch,  handling, 
and  recovery  system.  The  Navy  reported  that  it 
expects  the  results  of  now  completed  survivability 
testing  of  the  aluminum  structure  by  early  2017. 
Regarding  the  launch,  handling,  and  recovery 
system,  the  program  demonstrated  unmanned 
operations  during  LCS  8's  acceptance  trial,  but  has 
yet  to  receive  Navy  certification  to  conduct  manned 
operations  as  intended. 

Design  and  Production  Maturity 

The  LCS  4  survivability  trial  in  January  2016 
revealed  weaknesses  in  the  Independence  variant 
design,  according  to  the  Director,  Operational  Test 
and  Evaluation  (DOT&E).  In  July  2016,  LCS  6 
completed  shock  trials  in  accordance  with  the 
DOT&E  approved  plan.  This  trial  was  conducted  at 
a  reduced  severity  due  to  serious  concerns  about 
the  potential  for  damage  to  the  ship.  LCS  5  did  not 
complete  the  entire  shock  trial  because  the  Navy 
stopped  testing  in  September  201 6  due  to  concerns 
with  the  shock  environment,  personnel,  and 
equipment.  The  Navy  and  DOT&E  disagree  on  the 
need  to  complete  this  trial.  The  program  now 
expects  results  of  rough  water  trials — testing  that 
occurred  and  resulted  in  damage  on  both  designs 
several  years  ago — by  June  2017. 

Since  December  2015,  five  of  the  eight  delivered 
LCS — ships  of  both  variants — have  suffered 
engineering  casualties,  which  the  Navy  attributes  to 
shortfalls  in  crew  training,  seaframe  design,  and 
construction  quality.  According  to  the  Navy,  these 
failures  have  resulted  in  substantial  downtime  and 
costs  for  repairs  or  replacements.  We  have  found 
the  Navy  is  responsible  for  paying  for  the  vast 
majority  of  these  types  of  damage,  deficiencies,  and 
defects  on  ships  already  delivered.  While 
addressing  deficiencies  in  the  designs  of  each 
variant  to  increase  the  operational  availability  of  the 
ships  in-service,  the  Navy  is  also  working  to 
incorporate  changes  on  follow-on  ships.  The  Navy 
plans  to  make  improvements  to  LCS  either  during 
construction  or  sometime  after  delivery,  if  funding  is 
available. 


To  date,  nine  LCS  have  been  delivered  and  13  are 
in  various  phases  of  construction.  In  201 5,  the  Navy 
provided  the  LCS  shipbuilders  schedule  relief; 
however,  even  with  modified  ship  delivery  dates, 
both  shipbuilders  continue  to  deliver  LCS 
seaframes  significantly  behind  the  adjusted 
schedule.  Program  officials  recently  reported  the 
shipyards  would  not  deliver  four  LCS  in  fiscal  year 
2016  as  planned.  In  addition  to  lagging  schedule 
performance,  the  shipyards  continue  to  deliver 
seaframes  in  excess  of  cost  targets. 

Other  Program  Issues 

Following  a  pattern  of  LCS  engineering  casualties, 
in  February  2016,  the  Navy  initiated  a  program 
review  to  assess,  among  other  things,  LCS  crewing, 
training,  and  maintenance.  Recommended  actions 
included,  returning  to  a  "Blue/Gold"  crew  rotation 
model;  merging  the  seaframe  and  mission  package 
crew  into  a  single,  approximately  70-person  crew 
focused  on  a  single  mission  area;  and  designating 
LCS  1-4  as  test  ships  to  support  testing  between 
fiscal  years  2017  and  2022.  In  merging  the 
seaframe  and  mission  package  crew,  the  Navy 
acknowledged  that  switching  the  LCS  mission 
package — once  a  key  building  block  of  the  LCS 
concept — will  occur  less  often  than  originally 
conceived. 

Program  Office  Comments 

In  addition  to  providing  technical  comments,  the 
program  office  noted  as  of  January  2017,  there  are 
nine  LCS  in  the  Fleet,  with  another  17  on  contract. 
By  2018,  LCS  will  be  the  second  largest  surface 
ship  class  in  the  Navy.  Program  officials  reported 
the  LCS  design  is  stable,  meets  all  validated  and 
approved  requirements,  and  is  in  full  serial 
production  at  both  shipyards.  Program  officials  also 
reported  the  LCS  program  is  on  budget  and  below 
the  congressional  cost  cap  and  hull  over  hull 
performance  continues  to  improve,  stabilizing  the 
production  cycle.  Program  officials  stated  LCS  5 
and  6  successfully  met  all  test  objectives  of  the 
approved  shock  trial  test  plan,  demonstrating  the 
ability  of  both  variants  to  survive  the  effects  of 
underwater  shock  associated  with  the  close- 
proximity  detonation  of  a  10,000  pound  charge.  The 
program  office  stated  they  have  completed  required 
testing  and  are  incorporating  lessons  learned  into 
future  LCS  and  frigates. 


Page  106 


GAO-1 7-333SP  Assessments  of  Selected  Weapon  Programs 


Lead  Component:  Navy 


Common  Name:  LCS  Packages 


Littoral  Combat  Ship-Mission  Modules  (LCS  Packages) 


The  Littoral  Combat  Ship  (LCS)  will  provide  mine 
countermeasures  (MCM),  surface  warfare  (SUW), 
and  antisubmarine  warfare  (ASW)  capability  using 
mission  packages.  Packages  include  weapons  and 
sensors  launched  and  recovered  from  LCS 
seaframes.  The  Navy  plans  to  deliver  these 
capabilities  incrementally  and  has  set  interim 
requirements  that  are  below  the  baseline 
requirements  for  some  increments.  We  assessed 
mission  packages'  progress  against  the  threshold 
requirements  that  define  the  baseline  capabilities 
currently  expected  for  each  package. 


Source:  U.  S.  Navy. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  in  millions) 

Prime  contractor:  Northrop  Grumman 
Systems 

Program  office:  Washington  Navy  Yard, 

Research  and  development  cost 

As  of 
08/2007 

NA 

Latest 

09/2016 

$2,611.6 

Percent 

change 

NA 

DC 

Procurement  cost 

$3,535.4 

$4,456.3 

26.0% 

Funding  needed  to  complete: 

Total  program  cost 

NA 

$7,100.7 

NA 

R&D:  $518.3  million 

Program  unit  cost 

NA 

$110,949 

NA 

Procurement:  $3,657.4  million 

Total  quantities 

64 

64 

0.0% 

Total  funding:  $4,192.9  million 

Acquisition  cycle  time  (months) 

NA 

NA 

NA 

Procurement  quantity:  43 

The  current  estimate  does  not  include  $3.6  billion  of  procurement 
and  modernization  of  mission  systems. 

unding  for  life  cycle  replacement 

The  Navy  has  accepted  delivery  of  15  mission 
packages  without  demonstrating  whether  any 
meet  full  threshold  capability  requirements  for 
seaframes.  These  deliveries  followed  Navy 
decisions  to  design  and  produce  mission  package 
systems  prior  to  maturing  critical  technologies. 
The  Navy  has  recently  revised  its  approach  to 
develop  the  MCM  package  following  the 
discovery  of  significant  reliability  issues  during  a 
series  of  development  tests  in  fall  2015.  The  Navy 
intends  to  test  the  revised  MCM  package  against 
threshold  requirements  by  late  fiscal  year  2020.  A 
portion  of  the  SUW  package  has  met  its  interim 
requirements  on  one  of  the  seaframe  variants.  To 
meet  threshold  requirements,  the  Navy  is 
currently  modifying  an  Army  missile  it  plans  to  test 
in  fiscal  2018.  ASW  package  initial  capability  has 
been  delayed  due  to  weight  and  contracting 
issues  and  is  now  scheduled  for  fiscal  year  2019. 
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Attainment  of  Product  Knowledge 

As  of  January  2017 

STATUS  CURRENT  1 
AT  STATUS  1 

1  Resources  and  requirements  match 

•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

DEVELOPMENT 

START 

O  O 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

O 

O 

•  Complete  a  system-level  preliminary  design  review 

O 

• 

1  Product  design  is  stable 

•  Release  at  least  90  percent  of  design  drawings 

DESIGN  REVIEW  j 

....  # 

•  Test  a  system-level  integrated  prototype 

o 

• 

1  Manufacturing  processes  are  mature 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

9  or  critical  processes  are  in  statistical  control 

PRODUCTION 

START 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 

^  Knowledge  attained  ■  ■  ■  ■  Information  not  available 
O  Knowledge  not  attained  Not  applicable 


GAO-1 7-333SP  Assessments  of  Selected  Weapon  Programs 


Common  Name:  LCS  Packages 


LCS  Packages  Program 

Mine  Countermeasures  (MCM) 

The  Navy  designed  and  produced  MCM  mission 
package  systems  prior  to  maturing  critical 
technologies.  The  Navy  accepted  seven  MCM 
packages  without  demonstrating  they  meet 
threshold  performance  requirements  and,  is  now 
replacing  a  key  system — the  remote  multi-mission 
vehicle  (RMMV).  There  are  six  MCM  systems  (Near 
Surface  Detection,  Airborne  Mine  Neutralization, 
Remote  Minehunting,  Coastal  Mine 
Reconnaissance,  Buried  Minehunting,  and 
Unmanned  Mine  Sweeping)  the  Navy  plans  to 
assemble  and  fully  test  in  fiscal  year  2020.  After  the 
Navy  suspended  developmental  testing  in  October 
2015  following  the  discovery  of  significant  reliability 
issues,  it  studied  the  package  and  revised  its 
approach.  The  Navy  is  now  replacing  the  RMMV, 
which  towed  the  AQS-20A  sonar,  with  an  unmanned 
boat.  The  new  boat  rides  on  the  surface  of  the  water 
as  opposed  to  the  semi-submersible  RMMV. 
Program  officials  state  the  boat  will  be  easier  to 
launch  and  recover  but  could  be  susceptible  to 
wave-movement,  which  may  make  it  more  difficult 
to  find  mines.  The  Near  Surface  Detection  Module 
and  Airborne  Mine  Neutralization  Modules  achieved 
initial  capability  in  2016.  The  remaining  systems  are 
still  in  development  and  are  planned  to  be  tested 
over  the  next  several  years. 

Surface  Warfare  (SUW) 

The  Navy  designed  and  produced  SUW  mission 
package  systems  prior  to  demonstrating  the 
maturity  of  key  systems  leading  to  changes  and 
delays  to  the  SUW  package.  The  Navy  has 
accepted  eight  SUW  packages  with  no  deliveries 
planned  for  fiscal  year  2017.  One  package  currently 
consists  of  two  30  millimeter  guns,  an  armed 
helicopter,  and  two  rigid  hull  inflatable  boats.  In 
August  2014,  the  Navy  found  that  the  current 
package  met  interim  performance  requirements  on 
the  Freedom  variant  and,  in  2015,  the  Navy  tested 
this  part  of  the  package  on  the  Independence 
variant.  To  meet  threshold  requirements  a  surface- 
to-surface  missile  is  required.  According  to  program 
officials,  initial  missile  demonstrations  were 
successful,  but  operational  testing  was  delayed  by 
about  a  year  to  fiscal  year  201 8  due  to  ship 
integration  issues. 


Antisubmarine  Warfare  (ASW) 

The  Navy  reconfigured  the  ASW  package  after 
determining  planned  systems  would  not  provide 
adequate  capability.  According  to  the  Navy,  the 
ASW  systems  are  mature  as  they  have  been  fielded 
by  U.S.  Navy  and  foreign  navies.  Navy  program 
officials  stated  that  the  package’s  weight  issues 
have  been  resolved,  and  the  Navy  has  purchased 
an  initial  ASW  package  to  be  used  for  testing.  The 
Navy  is  now  planning  to  meet  the  threshold 
requirement  for  ASW  in  fiscal  year  2019,  a  2-year 
delay  from  last  year's  estimate. 

Other  Program  Issues 

The  Navy  will  not  achieve  the  capability  to  meet 
threshold  requirements  for  all  three  of  the  mission 
packages  until  late  fiscal  year  2020,  by  which  time  it 
plans  to  have  taken  delivery  of  24  ships.  Starting  in 
201 8,  the  Navy  plans  to  modify  LCS  as  a  frigate  and 
permanently  install  most  of  the  ASW  and  SUW 
mission  packages.  These  changes  have,  to  date, 
not  deterred  the  Navy  from  its  plans  to  purchase  64 
mission  packages. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
Navy  reported  it  is  delivering  operationally  effective 
mission  package  capability  to  the  fleet  as  it  mature 
increments.  The  Navy  stated  it  is  purchasing  the 
quantity  of  mission  systems  and  packages  needed 
for  system  integration,  crew  training,  developmental 
and  operational  testing,  and  LCS  deployments.  The 
Navy  reports  it  is  purchasing  the  systems  in 
accordance  with  relevant  laws  and  DOD 
regulations.  The  SUW  package  achieved  initial 
capability  in  fiscal  2015  and  will  meet  requirements 
with  the  surface  to  surface  missile  module  in  fiscal 
2018.  ASW  capability  is  planned  to  have  an  initial 
capability  and  meet  requirements  in  fiscal  2019.  The 
MCM  package  is  delivering  systems  as  they  mature. 
Due  to  reliability  of  the  RMMV,  the  Navy  reports  it  is 
restructuring  the  MCM  package  to  perform  the 
minehunting  mission  with  a  different  vehicle.  The 
MCM  package  is  planned  to  achieve  an  initial 
capability  in  fiscal  year  2020.  The  Navy  reported  it 
intends  to  adjust  the  program's  package  quantities 
in  2017  to  support  changes  to  the  LCS  and  frigate 
programs. 
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GAO-1 7-333SP  Assessments  of  Selected  Weapon  Programs 


Lead  Component:  Navy 


Common  Name:  MQ-4C  Triton 


MQ-4C  Triton  Unmanned  Aircraft  System  (MQ-4C  Triton) 


The  Navy's  MQ-4C  Triton  is  intended  to  provide 
persistent  maritime  intelligence,  surveillance,  and 
reconnaissance  (ISR)  data  collection  and 
dissemination  capability.  Triton  is  planned  to  be  an 
unmanned  aircraft  system  operated  from  five  land- 
based  sites  worldwide  as  part  of  a  family  of  maritime 
patrol  and  reconnaissance  systems.  Based  on  the 
Air  Force's  RQ-4B  Global  Hawk  air  vehicle,  Triton  is 
part  of  the  Navy's  plan  to  recapitalize  its  airborne 
ISR  assets  by  the  end  of  the  decade. 


Source:  Northrop  Grumman. 


sncept  System  development 


▲  ▲ 

Program/  Design 

development  start  review 

(4/08)  (2/11) 


▲  ▲ 

Low-rate  GAO 

decision  review 

(9/16)  (1/17) 


A  ▲  ▲ 

Start  Initial  Full-rate 

operational  test  capability  decision 

(11/20)  (2/21)  (5/21) 


Program  Essentials 

Prime  contractor:  Northrop  Grumman 
Systems  Corporation 
Program  office:  Patuxent  River,  MD 
Funding  needed  to  complete: 

R&D:  $120.3  million 
Procurement:  $7,951 .8  million 
Total  funding:  $8,165.8  million 
Procurement  quantity:  63 


Program  Performance  (fiscal  year  2017  dollars  in  millions) 


As  of 

Latest 

Percent 

02/2009 

09/2016 

change 

Research  and  development  cost 

$3,407.8 

$4,084.4 

19.9% 

Procurement  cost 

$10,113.1 

$8,733.4 

-13.6% 

Total  program  cost 

$13,935.8 

$13,123.1 

-5.8% 

Program  unit  cost 

$199,083 

$187,473 

-5.8% 

Total  quantities 

70 

70 

0.0% 

Acquisition  cycle  time  (months) 

92 

154 

67.4% 

R&D  costs  of  $681 .2  million  and  procurement  costs  of  $674  million  associated  with  phased 
modifications  to  update  sensor  and  system  performance  are  not  included  in  the  cost  data. 


The  MQ-4C  Triton  program  entered  low-rate 
production  in  September  2016  with  a  mature 
critical  technology  and  a  stable  design,  but  prior 
to  demonstrating  full  control  of  critical 
manufacturing  processes.  Developmental 
challenges  caused  the  Navy  to  delay  production 
start  by  over  3  years,  and  full-rate  production  has 
been  delayed  from  September  201 8  to  May  2021 . 
The  program  is  developing  new  capabilities  for 
Triton,  including  enhanced  intelligence  sensors 
and  an  aircraft  avoidance  capability.  The  program 
plans  to  integrate  these  into  production  aircraft 
beginning  in  fiscal  years  2020  and  2024, 
respectively.  Testing  delays  contributed  to 
program  decisions  to  concurrently  develop 
software  packages  needed  for  Triton’s  baseline 
configuration  and  future  upgrades.  The  baseline 
and  enhanced  intelligence  capabilities  are  now 
scheduled  to  be  demonstrated  prior  to  full-rate 
production. 

Page  109 


Attainment  of  Product  Knowledge 


As  of  January  2017 

STATUS  CURRENT 
AT  STATUS 

I  Resources  and  requirements  match 

•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

DEVELOPMENT 

START 

• 

• 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

o 

• 

•  Complete  a  system-level  preliminary  design  review 

o 

• 

1  Product  design  is  stable 

•  Release  at  least  90  percent  of  design  drawings 

DESIGN  REVIEW 

O 

• 

•  Test  a  system-level  integrated  prototype 

O 

• 

1  Manufacturing  processes  are  mature 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

9  or  critical  processes  are  in  statistical  control 

PRODUCTION 

START 

O 

o 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

• 

• 

•  Test  a  production-representative  prototype 

o 

o 

^  Knowledge  attained  Information  not  available 

o  Knowledge  not  attained  Not  applicable 


GAO-1 7-333SP  Assessments  of  Selected  Weapon  Programs 


Common  Name:  MQ-4C  Triton 


MQ-4C  Triton  Program 

Technology,  Design,  and  Production  Maturity 

The  Triton  program  entered  low-rate  production  in 
September  2016.  At  production  start,  the  program’s 
only  critical  technology  for  its  baseline 
configuration — the  hydrocarbon  sensor — was 
mature,  and  the  Triton  system  design  was  stable. 
According  to  best  practices  criteria,  technology 
maturity  and  design  stability  should  occur  prior  to 
development  start  and  critical  design  review, 
respectively,  neither  of  which  the  Triton  program 
achieved. 

Further,  Triton  production  began  before  the 
contractor  brought  its  manufacturing  processes  for 
wing  production  under  control,  which  was 
inconsistent  with  best  practices.  The  Defense 
Contract  Management  Agency  previously  identified 
quality  issues  with  the  wing  supplier  and  defects 
such  as  improperly  drilled  holes  and  gaps  in  the 
adhesive  that  bonds  the  wing  skin  to  the  wing 
frame.  Triton  wing  quality  problems  continue  to 
delay  wing  delivery  for  some  test  aircraft  being  built 
and  further  delays  could  impact  testing  schedules. 
According  to  the  program,  the  subcontractor 
increased  its  work  force  and  replaced  leadership  to 
expedite  wing  delivery  and  improve  its 
manufacturing  processes. 

Triton  developmental  challenges  caused  the  Navy 
to  delay  production  start  by  over  3  years.  In  order  to 
accommodate  cost  and  schedule  growth,  the  Navy 
delayed  the  start  of  full-rate  production — from 
September  201 8  to  May  2021 .  As  part  of  the  new 
schedule,  the  program  plans  to  increase  the 
quantity  of  aircraft  acquired  during  low-rate 
production  from  10  to  15,  increasing  the  risk  that 
more  vehicles  may  require  design  changes  prior  to 
demonstrating  Triton  capabilities. 

The  program  is  also  developing  new  Triton 
capabilities  it  will  implement  in  two  future  upgrade 
packages.  The  first  upgrade  will  include  enhanced 
intelligence  capabilities,  which  the  program  plans  to 
integrate  into  production  in  2020,  including  retrofits 
to  the  first  six  production  aircraft.  The  second 
upgrade  will  include  electronic  defense  upgrades 
and  an  aircraft  avoidance  radar  system.  This 
upgrade  package  is  scheduled  to  be  integrated  into 
the  production  line  for  the  19th  production  aircraft 
scheduled  to  be  delivered  in  fiscal  year  2024.  The 


Navy  originally  planned  to  include  the  aircraft 
avoidance  radar  in  the  baseline  configuration,  but 
deferred  this  due  to  technical  challenges. 

The  Triton  program  previously  planned  to  complete 
the  software  to  support  the  baseline  configuration 
before  initiating  software  development  for  the 
planned  upgrades.  However,  under  the  new 
schedule  the  program  will  begin  the  enhanced 
intelligence  software  upgrade  before  completing  the 
baseline  software,  thus  increasing  concurrency  risk. 
The  program  now  intends  to  demonstrate  the 
software  for  the  baseline  configuration,  along  with 
the  enhanced  intelligence  capabilities,  by  the  May 
2021  full-rate  production  date. 

Other  Program  Issues 

The  Navy  plans  to  deploy  aircraft  in  2018  that 
provides  intelligence  surveillance  and 
reconnaissance  capability,  but  some  capabilities 
such  as  high  resolution  radar  and  fire  detection 
have  been  deferred  to  future  software  builds  to 
address  cost  and  schedule  growth.  The  Navy  will 
field  them  as  an  early  operational  capability  and 
plans  to  support  the  deployment  with  operational 
testing  beginning  in  2017. 

Program  Office  Comments 

According  to  program  officials,  the  MQ-4C  Triton 
unmanned  aircraft  system  program  continues  to 
demonstrate  success  during  development  and  early 
operational  flight  and  ground  test.  Officials  noted 
modifications  to  install  the  baseline  early 
operational  capability  software  build  into  all  test 
assets  are  ongoing  in  support  of  flight  testing  to 
commence  in  the  second  quarter  of  fiscal  year 
2017.  Officials  further  noted  MQ-4C  Triton  low-rate 
initial  production  has  commenced  and  the  program 
is  planning  to  receive  two  additional  test  aircraft  in 
the  fourth  quarter  of  fiscal  year  201 7  to 
subsequently  support  an  operational  test  period. 


Page  110 


GAO-1 7-333SP  Assessments  of  Selected  Weapon  Programs 


Lead  Component:  Navy 


Common  Name:  MQ-8  Fire  Scout 


MQ-8  Fire  Scout 

The  Navy's  MQ-8  unmanned  aerial  vehicle  is 
intended  to  provide  real-time  imagery  and  data  in 
support  of  intelligence,  surveillance,  and 
reconnaissance  missions.  The  MQ-8  system  is 
comprised  of  one  or  more  air  vehicles  with  sensors, 
a  control  station,  and  ship  equipment  to  aid  in 
vertical  launch  and  recovery.  The  air  vehicle 
operates  from  ships  and  the  ground.  The  MQ-8  is 
intended  for  use  in  various  operations,  including 
surface,  anti-submarine,  and  mine  warfare,  and  it 
includes  B  and  C  variants.  We  assessed  the  latest 
variant,  the  MQ-8C. 

Source:  Northrop  Grumman  Systems  Corporation. 
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Production 


▲ 
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▲ 
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▲ 
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Design 

MQ-8C  rapid 

MQ-8C 

GAO 

Production 

MQ-8C  start 

MQ-8C 

start 

review 

deployment  start 

first  flight 

review 
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operational  test 

Initial  capability 

(1/00) 

(11/05) 

(4/12) 

(10/13) 

(1/17) 

(3/17) 

(5-9/17) 

(5/18) 

Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  in  millions) 

Prime  contractor:  Northrop  Grumman 

As  of 

Latest 

Percent 

Program  office:  Patuxent  River,  MD 

12/2006 

09/2016 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$648.6 

$1,246.3 

92.1% 

R&D:  $77.4  million 

Procurement  cost 

$1,825.0 

$1,689.0 

-7.4% 

Procurement:  $562.0  million 

Total  program  cost 

$2,836.8 

$2,935.4 

3.5% 

Total  funding:  $639.4  million 

Program  unit  cost 

$16,027 

$41,934 

161.6% 

Procurement  quantity:  11 

Total  quantities 

177 

70 

-60.5% 

Acquisition  cycle  time  (months) 

104 

170 

63.5% 

The  MQ-8C  program  has  mature  technologies 
and  a  stable  design.  The  MQ-8C  is  a  larger 
airframe  but  shares  technology  previously 
developed  for  the  MQ-8B.  The  MQ-8C  bypassed 
many  standard  acquisition  practices,  accepting 
risk  for  a  quicker  cycle  time,  because  it  was 
procured  under  the  rapid  deployment  capability 
process.  According  to  program  officials,  the 
program  plans  to  start  operational  testing  in 
spring  and  summer  2017  and  demonstrate  initial 
operational  capability  for  the  MQ-8C  in  May  2018. 
We  could  not  assess  production  process  maturity 
because  the  program  does  not  collect  statistical 
process  control  data  or  manufacturing  readiness 
levels.  The  Navy  restructured  the  program  in 
201 5  after  reporting  to  Congress  the  program  had 
incurred  unit  cost  growth  in  excess  of  statutory 
critical  thresholds. 
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Attainment  of  Product  Knowledge 

As  of  January  2017 

STATUS  CURRENT 
AT  STATUS 

I  Resources  and  requirements  match 

DEVELOPMENT 

•  Demonstrate  all  critical  technologies  are  very  close  to 

START 

final  form,  fit,  and  function  within  a  relevant  environment 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 

function  within  a  realistic  environment 

•  Complete  a  system-level  preliminary  design  review 

O 

• 

1  Product  design  is  stable 

DESIGN  REVIEW 

•  Release  at  least  90  percent  of  design  drawings 

O 

• 

•  Test  a  system-level  integrated  prototype 

O 

• 

1  Manufacturing  processes  are  mature 

PRODUCTION 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

START 

9  or  critical  processes  are  in  statistical  control 

■  ■  ■  ■ 

■  ■  ■  ■ 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

• 

• 

•  Test  a  production-representative  prototype 

• 

• 

0  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

O  Knowledge  not  attained  Not  applicable 

GAO-1 7-333SP  Assessments  of  Selected  Weapon  Programs 


Common  Name:  MQ-8  Fire  Scout 


MQ-8  Fire  Scout  Program 

Technology  and  Design  Maturity 

The  MQ-8C  relies  on  mature  technologies  common 
to  the  MQ-8B  and  has  a  stable  design.  According  to 
the  program  office,  the  MQ-8C  has  90  percent 
commonality  with  the  previously  developed  MQ-8B. 
The  primary  differences  between  the  two  are 
structural  modifications  to  accommodate  the  MQ- 
8C's  larger  airframe  and  fuel  system.  The  MQ-8C 
was  initiated  under  the  Navy's  rapid  deployment 
capability  acquisition  process,  which  program 
officials  stated  enabled  the  program  to  bypass  many 
standard  acquisition  practices  designed  to  reduce 
risk  in  favor  of  a  speedier  acquisition  cycle  time.  The 
Navy  completed  its  first  MQ-8C  flight  in  October 
201 3.  The  program  will  continue  with  developmental 
testing  and  transition  to  operational  testing  in  the 
spring  and  summer  2017.  Despite  being  a  separate 
increment  of  capability,  the  MQ-8C  did  not  have  a 
separate  development  start  decision  review  which, 
according  to  program  officials,  was  not  required. 

Production  Maturity 

The  program  awarded  a  contract  for  development 
and  production  of  the  MQ-8C  in  April  2012  after 
production  of  the  MQ-8B  variant  stopped  following 
delivery  of  30  aircraft.  As  of  September  2016,  the 
program  has  taken  delivery  of  19  MQ-8C  aircraft, 
which  program  officials  consider  low-rate  initial 
production  units.  An  additional  11  MQ-8C  aircraft 
are  under  contract,  including  one  fiscal  year  2017 
aircraft.  According  to  program  officials,  the  program 
plans  to  make  a  full-rate  production  decision, 
previously  scheduled  for  May  2016,  in  the  second 
quarter  of  fiscal  year  2017.  In  addition,  the  program 
reports  it  plans  to  start  operational  testing  in 
September  2017  and  demonstrate  initial  operational 
capability  in  May  2018. 

We  could  not  assess  whether  critical  manufacturing 
processes  were  in  control  as  the  program  does  not 
collect  data  on  statistical  manufacturing  process 
controls  or  assess  process  capabilities  using 
manufacturing  readiness  levels.  Rather,  the 
program  office  collects  metrics  on  the  status  of 
production  from  the  prime  contractor,  such  as 
discovery  of  manufacturing  defects.  Program 
officials  noted  the  MQ-8C  has  a  commercial 
airframe  procured  by  the  government  and  provided 
directly  to  the  prime  contractor  as  government- 
furnished  equipment  for  conversion  to  an  MQ-8C. 


The  prime  contractor  is  responsible  for  integration  of 
the  avionics  and  working  with  the  aircraft 
manufacturer  to  modify  the  commercial  airframe 
with  increased  capacity  fuel  tanks. 

Other  Program  Issues 

In  June  201 4,  the  program  was  certified  as  essential 
to  national  security  and  allowed  to  progress  after 
reporting  to  Congress  it  had  incurred  unit  cost 
growth  in  excess  of  statutory  critical  thresholds. 
Program  officials  told  us  the  Navy  restructured  the 
program  in  2014,  with  revised  quantities  of  both 
variants,  because  the  MQ-8C  had  more  endurance 
than  the  MQ-8B,  which  reduced  the  total  quantity  of 
MQ-8  units  required  to  support  the  Littoral  Combat 
Ship.  Following  this  determination,  program  officials 
said  they  stopped  production  of  the  MQ-8B  and 
moved  forward  with  only  producing  the  MQ-C. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  noted  the  MQ-8C  program  is 
executing  with  no  significant  issues.  The  program 
further  noted  it  awaits  availability  of  the  Littoral 
Combat  Ship  to  complete  MQ-8C  testing,  including 
planned  start  of  operational  testing  in  spring  and 
summer  2017.  The  program  office  stated  the  MQ- 
8B  continues  deployments  aboard  the  Littoral 
Combat  Ship,  including  the  first  deployment  with  an 
upgraded  radar.  In  addition,  the  program  office 
provided  technical  comments,  which  were 
incorporated  where  deemed  appropriate. 
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Lead  Component:  Navy 


Common  Name:  NGJ  Increment  1 


Next  Generation  Jammer  Increment  1 

The  Navy's  Next  Generation  Jammer  (NGJ)  is  being 
developed  as  an  external  jamming  pod  system  fitted 
on  EA-18G  Growler  aircraft.  It  is  expected  to 
replace  the  ALQ-99  tactical  jamming  system  and 
provide  enhanced  airborne  electronic  attack 
capabilities  to  disrupt  adversaries'  use  of  the 
electromagnetic  spectrum.  The  Navy  plans  to  field 
capabilities  in  three  increments  for  different  radio 
frequency  ranges,  beginning  with  Increment  1  (mid¬ 
band)  in  2021 ,  with  Increments  2  and  3  (low-  and 
high-band)  to  follow.  We  assessed  Increments  1 
and  2. 


Source:  Raytheon  Corp. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017  dollars  in  millions) 

Prime  contractor:  Raytheon 

As  of 

Latest 

Percent 

Program  office:  Patuxent  River,  MD 

09/2016 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

NA 

$3,516.3 

NA 

R&D:  $2,207.2  million 

Procurement  cost 

NA 

$4,074.7 

NA 

Procurement:  $4,074.7  million 

Total  program  cost 

NA 

$7,598.2 

NA 

Total  funding:  $6,289.0  million 

Program  unit  cost 

NA 

$56,283 

NA 

Procurement  quantity:  131 

Total  quantities 

NA 

135 

NA 

Acquisition  cycle  time  (months) 

NA 

98 

NA 

NGJ  Increment  1  entered  development  in  April 
2016  with  its  critical  technologies  nearing 
maturity.  Consistent  with  best  practices  and 
statutory  requirements,  the  program  completed  a 
preliminary  design  review  prior  to  starting 
development.  Program  officials  state  NGJ  is  on 
track  to  hold  its  critical  design  review  (CDR)  in 
early  2017.  As  of  November  2016,  the  contractor 
had  released  almost  76  percent  of  its  design 
drawings.  The  Naval  Air  Systems  Command 
benchmark  for  released  drawings  at  critical 
design  review  is  80  percent,  which  is  short  of  the 
90  percent  threshold  under  GAO  identified  best 
practices.  The  program's  main  risks  include 
meeting  weight  and  power  requirements  and 
integrating  the  jamming  pods  with  the  aircraft. 
NGJ  is  participating  in  a  pilot  program  that  allows 
it  to  streamline  and  tailor  some  acquisition 
processes. 


PRODUCTION 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 
9  or  critical  processes  are  in  statistical  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 

^  Knowledge  attained  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  NGJ  Increment  1 


NGJ  Increment  1  Program 

Technology  Maturity 

The  NGJ  program  entered  system  development  in 
April  2016,  with  its  seven  critical  technologies 
approaching  maturity.  In  November  2015,  the 
program  conducted  an  assessment  of  these 
technologies,  which  includes  two  separate  arrays — 
each  with  different  transmit/receive  modules, 
circulators,  and  apertures — as  well  as  a  power 
generation  system.  All  technologies  were  assessed 
as  nearing  maturity  based  on  subsystem  prototype 
testing  conducted  prior  to  starting  development. 

Design  and  Production  Maturity 

Program  officials  stated  NGJ  is  on  track  to  hold  its 
CDR  in  March  201 7,  but  the  program  is  not 
following  GAO  identified  best  practices  for  this 
milestone.  As  of  November  201 6,  the  contractor  had 
released  about  76  percent  of  design  drawings.  This 
positions  the  program  to  meet  the  Naval  Air 
Systems  Command  benchmark  of  releasing  80 
percent  of  its  design  drawings  by  CDR.  However, 
GAO  identified  best  practices  call  for  releasing  90 
percent  of  drawings  by  CDR.  The  program  also 
does  not  plan  to  test  an  early  system  prototype  of 
the  jamming  pod  before  its  CDR.  GAO  identified 
best  practices  recommend  such  testing  prior  to  this 
key  review  to  help  determine  whether  the  design  will 
work  as  intended.  The  first  test  of  a  fully  functional 
jamming  pod  is  planned  for  May  2019,  4  months 
before  the  program's  planned  start  of  production. 

However,  program  officials  stated  that  the  significant 
prototyping  and  associated  testing  conducted  to 
date  address  NGJ's  principal  risks,  which  include 
weight,  integration  complexity,  and  transmit  power. 
The  program  recently  addressed  the  severity  of  its 
power  risk  by  taking  several  mitigation  actions.  In 
addition,  the  program  is  managing  software  risk 
through  modeling  and  simulation.  The  program  is 
consulting  with  systems  engineering  experts  from 
the  Office  of  the  Secretary  of  Defense  for  input 
regarding  NGJ  software  metrics. 

Regarding  production  maturity,  NGJ  officials  stated 
they  plan  to  identify  critical  manufacturing 
processes  and  key  product  characteristics  at  CDR. 
NGJ  officials  also  stated  they  have  begun  assessing 
manufacturing  readiness  levels. 


Other  Program  Issues 

In  September  2015,  the  Under  Secretary  of  Defense 
for  Acquisition,  Technology  and  Logistics  (USD 
AT&L)  approved  NGJ  as  the  first  program  in  the 
Skunk  Works  pilot,  which  aims  to  eliminate  non¬ 
value  added  processes  in  order  to  deliver 
capabilities  on  time  and  within  budget.  NGJ  officials 
stated  they  had  direct  access  to  the  USD  AT&L  and 
other  senior  acquisition  officials,  and  some  approval 
authorities  were  delegated  to  lower  levels.  For 
example,  the  contract  award  authority  was 
delegated  down  to  Naval  Air  Systems  Command, 
which  officials  said  saved  2  to  3  months  of  time  in 
the  contract  negotiation  process.  NGJ  officials  said 
the  Skunk  Works  designation  was  beneficial  to  the 
program. 

Studies  for  NGJ  Increment  2,  a  separately-managed 
program  for  low-band  jamming  that  had  been 
planned  for  fiscal  years  2014  and  2015,  were 
postponed  to  fiscal  year  2016  for  funding  reasons. 
As  a  result,  program  officials  stated  the  Navy 
pushed  back  Increment  2's  planned  entry  into 
development  from  fiscal  year  2018  to  2020, 
although  program  officials  stated  this  date  may 
change  as  the  studies  progress.  These  design  trade 
studies  are  intended  to  evaluate  the  maturity  of 
technologies  and  refine  requirements. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
NGJ  program  stated  NGJ  Increment  1  entered 
system  development  in  April  2016  after  successfully 
meeting  required  criteria.  NGJ  met  required 
technology  maturity  for  seven  critical  technologies 
and  established  a  program  baseline  after 
completion  of  preliminary  design  review  in  October 
2015.  As  of  November  2016,  the  contractor  had 
released  76  percent  of  the  design  drawings  and  is 
on  track  to  exceed  the  Naval  Air  Systems 
Command  best  practice  of  80  percent  completed  by 
critical  design  review.  The  program  completed 
significant  prototyping  of  key  systems  and 
subsystems  prior  to  system  development,  which 
reduced  risk  to  acceptable  levels  for  system 
development  as  determined  by  the  milestone 
decision  authority.  According  to  program  officials, 
NGJ's  extensive  use  of  contractor  and  government 
systems  integration  labs,  along  with  the  use  of  test 
chambers  to  evaluate  pod  performance,  will 
significantly  reduce  risk  prior  to  the  program’s 
planned  start  of  production. 


Page  114 


GAO-1 7-333SP  Assessments  of  Selected  Weapon  Programs 
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Common  Name:  OASuW  Inc  1 


Offensive  Anti-Surface  Warfare  Increment  1  (OASuW  Inc  1) 


The  Offensive  Anti-Surface  Warfare  Increment  1 
(OASuW  Inc  1)  is  a  Navy-led  program  to  develop  an 
air-launched,  long-range,  anti-surface  warfare 
missile  to  address  an  urgent  operational  need.  The 
program  is  using  an  accelerated  acquisition 
approach  and  leveraging  previous  technology 
demonstration  efforts.  It  plans  to  field  an  early 
operational  capability  on  Air  Force  B-1  bombers  in 
2018  and  Navy  F/A-18  aircraft  in  2019.  DOD  is 
developing  specifications  for  Increment  2  to  address 
the  threat  in  2024  and  beyond.  We  assessed 
Increment  1 . 


Source:  U.S.  Navy. 
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Program  Performance  (fiscal  year  2017  dollars  in  millions) 


Program  Essentials 

Prime  contractor:  Lockheed  Martin 
Proqram  office:  Patuxent  River  NAS, 
MD 

Funding  needed  to  complete: 

R&D:  $494.7  million 
Procurement:  $341.0  million 
Total  funding:  $835.7  million 
Procurement  quantity:  110 


Research  and  development  cost 
Procurement  cost 
Total  program  cost 
Program  unit  cost 
Total  quantities 

Acquisition  cycle  time  (months) 


As  of 

Latest 

Percent 

03/2016 

08/2016 

change 

$1,223.8 

$1,154.3 

-5.7% 

$304.4 

$341 .0 

12.0% 

$1,528.3 

$1,495.3 

-2.2% 

$12,325 

$12,256 

-0.6% 

124 

122 

-1 .6% 

43 

42 

-2.3% 

The  OASuW  Inc  1  program  received  approval  to 
enter  production  in  December  201 6  with  its  critical 
technologies  mature,  a  stable  hardware  design, 
and  demonstrated  production  processes. 
Flowever,  technology  maturity  has  not  been 
confirmed  through  independent  assessment,  and 
flight  tests  will  not  occur  until  after  production 
begins.  Software  development  challenges  have 
delayed  aspects  of  subsystem  testing  but  are  not 
assessed  as  presenting  an  overall  schedule  risk. 
The  program’s  accelerated  acquisition  approach, 
designed  to  address  an  urgent  operational  need, 
includes  concurrency  between  developmental 
testing  and  production,  which  increases  the  risk  of 
late  design  changes  and  costly  retrofits  after 
production  has  started.  The  program  seeks  to 
mitigate  risks  by  leveraging  design  commonalities 
with  a  missile  already  in  production  and  limiting 
the  number  of  missiles  purchased. 
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Attainment  of  Product  Knowledge 


As  of  January  2017 


Resources  and  requirements  match 


STATUS  CURRENT 
AT  STATUS 


Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 


DEVELOPMENT 

START 


O 


Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 


o 


Complete  a  system-level  preliminary  design  review 


Product  design  is  stable 


Release  at  least  90  percent  of  design  drawings 
Test  a  system-level  integrated  prototype 


DESIGN  REVIEW 


o  o 


Manufacturing  processes  are  mature 


Demonstrate  Manufacturing  Readiness  Level  of  at  least 
9  or  critical  processes  are  in  statistical  control 


PRODUCTION 

START 


o  o 


Demonstrate  critical  processes  on  a  pilot  production  line 
Test  a  production-representative  prototype 


o  o 


9  Knowledge  attained 
O  Knowledge  not  attained 


Information  not  available 
Not  applicable 


Common  Name:  OASuW  Inc  1 


OASuW  Inc  1  Program 

Technology  Maturity 

While  the  most  recent  independent  technology 
readiness  assessment,  conducted  by  the  Navy  in 
August  2016,  assessed  the  program's  six  critical 
technologies  are  nearing  maturity,  in  the  past  year, 
the  OASuW  Inc  1  program  has  fully  matured  all  six 
of  its  critical  technologies.  The  program's  single, 
hardware-based  critical  technology — a  radio 
frequency  sensor — as  well  as  five  software-based 
ones — autorouter,  low  altitude  control  estimator, 
target  tracker,  electro-optical  algorithms,  and 
simultaneous  time  of  arrival  capbility — were 
demonstrated  in  a  relevant  environment.  According 
to  program  officials,  critical  technologies  were 
matured  further  through  a  combination  of  modeling 
and  simulation  and  flying  test  bed  events  that  did 
not  involve  launching  a  missile.  The  program  office 
assessed  all  critical  technologies  were  mature  at  its 
production  start  in  December  2016,  although  this 
was  not  confirmed  by  an  independent  assessment 
or  through  missile  flight  tests,  which  are  not  planned 
to  begin  until  the  fourth  quarter  of  fiscal  year  2017. 

Design  Maturity 

The  OASuW  Inc  1  hardware  design  is  stable,  but 
the  program  is  addressing  software  and 
manufacturing  challenges  that  have  delayed  testing 
used  to  demonstrate  the  design  will  work  as 
intended.  The  program  held  its  critical  design  review 
in  June  2016,  at  which  time  all  the  drawings 
expected  to  support  production  were  releasable  to 
manufacturers.  According  to  program  officials, 
nearly  all  subsystems  have  been  qualified,  which 
means  they  have  been  tested  to  ensure  that  they 
can  meet  requirements,  but  software  challenges 
delayed  some  subsystem  testing.  With  hardware 
design  stable,  officials  stated  remaining  software 
challenges  can  be  addressed  later  in  testing  without 
negatively  affecting  the  schedule.  Additionally, 
program  officials  noted  manufacturing  challenges 
resulted  in  test  unit  delivery  delays,  but  these  delays 
are  not  expected  to  affect  the  program's  testing 
schedule. 

Production  Maturity 

In  December  2016,  a  series  of  production  readiness 
reviews  found  that  critical  production  processes  for 
OASuW  Inc  1  were  demonstrated  in  a  production 
representative  environment  or  on  a  pilot  line.  The 
system  leverages  the  airframe  and  production 


facilities  of  the  Joint  Air-to-Surface  Standoff  Missile- 
Extended  Range  (JASSM-ER)  program,  which  the 
program  reports  will  decrease  production  risks.  The 
program  was  tracking  a  risk  related  to  the  JASSM- 
ER  fuze,  which  had  not  been  qualified  for  use  on 
Navy  weapons,  but  its  use  has  since  been  certified 
by  the  Navy.  The  first  missiles  to  support  an  early 
operational  capability  are  planned  for  delivery  in  the 
second  quarter  of  fiscal  year  201 8. 

Other  Program  Issues 

Maintaining  the  program's  schedule  is  the  primary 
concern  for  OASuW  Inc  1 ,  which  is  intended  to 
address  an  urgent  operational  need.  The  program's 
current  accelerated  acquisition  approach  consists  of 
decision  points  that  align  with  key  systems 
engineering  reviews,  test  events,  contract  actions, 
and  fielding  decisions.  The  approach  requires 
concurrency  between  developmental  testing  and 
initial  production.  Our  past  work  has  shown  that 
beginning  production  before  demonstrating  that  a 
design  is  stable  and  will  work  as  intended  increases 
the  risk  of  discovering  deficiencies  during 
production  that  could  require  costly  design  changes, 
modifications,  and  retrofits.  The  program  has 
accepted  this  risk  and  mitigated  it,  in  part,  by  limiting 
the  program  to  the  1 1 0  missiles  needed  to  provide  a 
capability  until  the  second  increment  is  fielded. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment, 
program  officials  noted  OASuW  Inc  1  is  an 
accelerated  acquisition  program  intended  to  fill  an 
urgent  capability  requirement  in  the  maritime 
domain.  Accordingly,  schedule  is  elevated  as  a 
priority  with  associated,  increased  emphasis  on  risk 
management,  including  those  related  to  concurrent 
system  test  and  production  activities.  According  to 
the  program,  the  hardware  design  is  mature. 
Program  risk  assessments  provide  additional 
oversight  and  an  executive  steering  board  chaired 
by  the  senior  Navy  acquisition  authority  regularly 
reviews  technical  progress,  facilitates  risk 
management  activities,  and  mitigates  barriers  to 
progress.  According  to  the  program,  OASuW  Inc  1 
has  successfully  progressed  from  technology 
development  through  a  production  readiness 
decision  in  10  months,  and  the  program  remains  on 
track  for  early  operational  capability  fielding  in  the 
fourth  quarter  of  fiscal  year  201 8.  The  program  also 
provided  technical  comments,  which  were 
incorporated  where  deemed  appropriate. 
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Lead  Component:  Navy 


Common  Name:  SSBN  826 


SSBN  826  Columbia  Class  Ballistic  Missile  Submarine  (SSBN  826) 


The  Navy's  new  Columbia  Class  (SSBN  826)  will 
replace  the  current  fleet  of  Ohio  Class  SSBNs  as 
they  begin  to  retire  in  2027.  The  program  seeks  to 
provide  affordable  platforms  for  sea-based  strategic 
deterrence  that  will  remain  in  service  through  2080. 
The  Navy  began  technology  development  in 
January  2011  in  order  to  avoid  a  capabilities  gap 
between  the  Ohio  Class's  retirement  and  the 
fielding  of  a  replacement  system.  The  program 
plans  to  begin  detail  design  in  2017,  with  lead  ship 
construction  starting  in  fiscal  year  2021. 


Source:  ©  2012  General  Dynamics  Electric  Boat. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  in  millions) 

Prime  contractor:  General  Dynamics 

As  of 

Latest 

Percent 

Electric  Boat 

01/2017 

01/2017 

change 

Program  office:  Washington  Navy  Yard, 

Research  and  development  cost 

$12,648.1 

$12,648.1 

0.0% 

DC 

Procurement  cost 

$87,426.5 

$87,426.5 

0.0% 

Funding  needed  to  complete: 

Total  program  cost 

$100,221.9 

$100,221.9 

0.0% 

R&D:  $6,113.2  million 

Program  unit  cost 

$8,351 .825 

$8,351,825 

0.0% 

Procurement:  $87,426.5  million 

Total  quantities 

12 

12 

0.0% 

Total  funding:  $93,666.3  million 

Acquisition  cycle  time  (months) 

232 

238 

2.6% 

Procurement  quantity:  12 

The  Columbia  Class  program  entered  system 
development  in  2017  with  one  critical  technology 
that  has  not  yet  been  demonstrated  in  a  relevant 
environment  and  several  other  major 
development  efforts.  The  Navy  plans  to  complete 
detail  design  during  this  phase  concurrently  with 
efforts  to  reduce  program  cost  and  develop 
Columbia-class  technologies.  The  Navy 
established  an  aggressive  schedule  for 
construction  of  the  lead  submarine  and  is  working 
with  future  Columbia  Class  shipyards  to  identify 
production  efficiencies  needed  to  meet  this  goal. 


Attainment  of  Product  Knowledge 


As  of  January  2017 

STATUS  CURRENT 
AT  STATUS 

I  Resources  and  requirements  match 

•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

DETAIL  DESIGN 
CONTRACT 
AWARD 

O 

O 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

O 

O 

•  Complete  a  system-level  preliminary  design  review 

• 

• 

1  Product  design  is  stable 

•  Complete  basic  and  functional  design  to  include 

100  percent  of  3D  product  modeling 

FABRICATION 

START 

^  Knowledge  attained  ■  ■  ■  ■  Information  not  available 
O  Knowledge  not  attained  Not  applicable 
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Common  Name:  SSBN  826 


SSBN  826  Program 

Technology  Maturity 

The  Columbia  Class  program's  most  recent 
Technology  Readiness  Assessment  (TRA)  identifies 
only  two  critical  technologies — one  carbon  dioxide 
removal  system  and  one  major  technical  feature  of 
the  stern.  The  Navy  has  subsequently  determined 
the  carbon  dioxide  removal  system  is  no  longer  a 
critical  technology  based  on  testing  in  a  relevant 
environment.  Only  the  stern  feature  remains  as  a 
critical  technology,  and  the  Navy  does  not  expect  to 
start  detail  design  for  this  technology  until  fiscal  year 
2018.  According  to  DOD's  guidance  for  conducting 
TRAs,  a  technology  is  critical  if  it  may  pose  major 
technological  risk  during  development,  particularly 
during  the  Engineering  and  Manufacturing 
Development  (EMD)  phase  of  acquisition.  Prior  to 
starting  EMD,  the  program  obtained  a  waiver  as 
required  by  statute  in  order  to  proceed  with 
technology  that  has  not  been  demonstrated  in  a 
relevant  environment.  As  a  result,  the  program  will 
continue  technology  development  activities 
concurrent  with  detail  design-this  strategy  is 
inconsistent  with  best  practices. 

Design  Maturity 

The  Navy  plans  to  include  up  to  70  percent  of 
Virginia  Class  submarine  parts  and  components  in 
the  Columbia  Class  design.  It  is  also  designing  the 
Columbia  Class  to  accommodate  the  existing 
Trident  II  D-5  strategic  weapon  system,  and  has 
already  completed  design  work  for  the  first  article 
quad  pack  of  missile  tubes.  The  program  is  relying 
on  a  new  three  dimensional  (3-D)  computer  model 
design  tool,  which  it  anticipates  will  produce 
electronic  instructions  in  support  of  fabrication  and 
assembly  activities.  In  2016,  the  design  shipbuilder 
experienced  a  delay  in  issuing  early  design 
products  due  to  problems  with  this  software. 
According  to  the  design  shipbuilder,  it  has  largely 
resolved  the  issue  and  is  working  to  recover 
schedule.  The  program  aims  to  complete  83  percent 
of  the  3-D  design  model  prior  to  the  start  of  lead 
ship  construction  in  fiscal  year  2021 . 

Production  Maturity 

The  Navy  plans  to  award  a  contract  to  start  detail 
design  in  2017,  with  lead  ship  construction 
scheduled  to  begin  in  fiscal  year  2021 .  The  Navy 
expects  to  build  the  lead  ship  in  84  months  with 
follow-on  ships  progressively  decreasing  to  70 


months  construction  time.  This  schedule  is 
aggressive,  considering  that  it  is  approximately  the 
same  duration  as  the  lead  Virginia  class  submarine, 
even  though  the  Columbia  Class  is  over  two  times 
larger  and  the  first  ballistic  missile  submarine  built  in 
decades.  According  to  the  Navy,  a  decision  in  2012 
to  delay  construction  start  from  2018  to  2020 
eliminated  flexibility  to  accommodate  delays  during 
construction,  given  that  the  lead  ship's  first  strategic 
deterrent  patrol  remains  scheduled  for  2030.  In  an 
effort  to  gain  back  some  schedule  margin,  the  Navy 
is  working  with  the  future  Columbia  Class 
construction  shipyards  to  identify  production 
efficiencies  and  is  requesting  funding  authorities 
from  Congress  to  begin  some  construction  work 
prior  to  authorization  of  the  lead  ship.  Under  the 
Navy's  current  plans,  the  shipyards  will  construct 
two  submarines  per  year  (one  Virginia  Class  and 
one  Columbia  Class)  through  the  2030s. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  technical  comments,  which 
were  incorporated  where  deemed  appropriate. 
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Lead  Component:  Navy 


Common  Name:  SSC 


Ship  to  Shore  Connector  Amphibious  Craft  (SSC) 

The  Navy's  SSC  is  an  air-cushioned  landing  craft 
intended  to  transport  personnel,  weapon  systems, 
equipment,  and  cargo  from  amphibious  vessels  to 
shore.  SSC  is  the  replacement  for  the  Landing 
Craft,  Air  Cushion,  which  is  approaching  the  end  of 
its  service  life.  The  SSC  is  designed  to  deploy  in 
and  from  Navy  amphibious  ships  with  well  decks, 
such  as  the  LPD  17  class,  and  will  support  assault 
and  non-assault  operations.  The  program  entered 
system  development  in  July  2012  and  held  its  low- 
rate  production  decision  in  May  2015. 


Source:  U.S.  Navy. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017  dollars  in  millions) 

Prime  contractor:  Textron  Inc. 

As  of 

Latest 

Percent 

Program  office:  Washington,  DC 

07/2012 

10/2016 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$603.7 

$541.1 

-10.4% 

R&D:  $20.9  million 

Procurement  cost 

$3,664.0 

$3,444.3 

-6.0% 

Procurement:  $3,098.4  million 

Total  program  cost 

$4,287.9 

$4,000.2 

-6.7% 

Total  funding:  $3,134.0  million 

Program  unit  cost 

$58,739 

$54,798 

-6.7% 

Procurement  quantity:  64 

Total  quantities 

73 

73 

0.0% 

Acquisition  cycle  time  (months) 

135 

135 

0.0% 

SSC's  critical  technology  is  mature  and  its  design 
is  stable.  The  program  entered  low-rate 
production  in  May  2015  and  has  experienced 
some  delays  in  its  production  schedule.  It  now 
anticipates  delivery  of  the  initial  test  and  training 
craft  in  August  2017,  three  months  later  than  the 
time  of  our  last  review.  According  to  the  program, 
the  primary  cause  of  these  delays  is  its  propeller 
supplier,  whose  production  yields  have  not  been 
reliable  following  a  2015  fire  that  destroyed  its 
plant.  The  program  has  nine  craft  under  contract, 
with  four  already  under  construction.  It  is  seeking 
authority  to  use  what  it  calls  a  "block  buy  contract" 
for  fiscal  years  201 7  through  2018.  If  issues 
requiring  design  changes  are  identified  during 
developmental  testing — not  scheduled  for 
completion  until  December  2017 — the  program 
could  face  costly  rework  of  the  craft  already  in 
production. 
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Attainment  of  Product  Knowledge 


As  of  January  2017 

STATUS  CURRENT 
AT  STATUS 

I  Resources  and  requirements  match 

•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

DEVELOPMENT 

START 

• 

• 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

• 

• 

•  Complete  a  system-level  preliminary  design  review 

• 

• 

1  Product  design  is  stable 

•  Release  at  least  90  percent  of  design  drawings 

DESIGN  REVIEW 

O 

• 

•  Test  a  system-level  integrated  prototype 

O 

o 

1  Manufacturing  processes  are  mature 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

9  or  critical  processes  are  in  statistical  control 

PRODUCTION 

START 

O 

o 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

• 

• 

•  Test  a  production-representative  prototype 

o 

o 

^  Knowledge  attained  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  SSC 


SSC  Program 

Technology  and  Design  Maturity 

SSC's  one  critical  technology  is  mature.  Although 
the  SSC's  design  was  not  stable  at  its  design  review 
in  2014,  it  has  since  achieved  stability.  The  gearbox, 
which  had  shown  premature  wear,  passed  factory 
acceptance  testing  in  fall  2016.  However,  further 
testing  identified  design  issues  with  high  speed 
bearings  within  the  gearbox.  A  solution  has  been 
developed  and  will  be  tested  by  February  2017.  In 
May  2015,  the  Navy's  operational  test  agency 
reported  that  the  steep  angle  of  SSC's  loading  ramp 
may  create  operational  hazards  for  certain  vehicles. 
The  program  is  designing  a  new  approach  ramp  to 
mitigate  this  risk,  but  will  not  have  an  opportunity  to 
fully  assess  this  and  other  issues  until 
developmental  testing  is  completed  in  December 
2017. 

Production  Maturity 

The  program  entered  low-rate  production  in  May 
2015  after  demonstrating  that  all  materials, 
manpower,  tooling,  and  facilities  were  proven  and 
available  to  meet  the  low-rate  production  schedule, 
as  recommended  by  DOD  guidance.  However, 
according  to  best  practices,  programs  should 
demonstrate  manufacturing  processes  to  be  in 
statistical  control  prior  to  production  start. 
Specifically,  critical  processes  should  be  repeatable, 
sustainable,  and  consistently  producing  parts  within 
the  quality  standards. 

The  program  has  experienced  delays  in  its 
production  schedule.  Nine  SSC  craft  are  under 
contract,  with  four  already  under  construction.  It 
now  expects  delivery  of  the  initial  test  and  training 
craft  in  August  2017,  a  3-month  delay  since  our  last 
review.  Delivery  of  the  second  craft  also  slipped  by 
3  months  to  November  2017.  According  to  the 
program,  the  primary  cause  of  these  delays  is  the 
supply  of  propellers,  1 2  of  which  are  needed  for 
each  craft.  A  fire  destroyed  the  manufacturer's 
facility  in  2015  and  production  yield  from  the  new 
facility  has  been  unreliable,  which  officials  stated 
poses  risk  to  the  SSC  production  schedule. 

Delivery  of  the  test  craft  in  late  201 7  will  offer  the 
first  opportunity  to  demonstrate  that  the  SSC  design 
meets  requirements  and  that  no  rework  is  needed. 
Testing  of  this  craft  will  occur  while  the  contractor 
produces  eight  other  SSC  craft.  Our  previous  work 


has  found  that  concurrent  testing  and  production 
increases  the  risk  of  discovering  deficiencies  that 
could  require  costly  design  changes  and 
modifications  to  units  already  produced. 

Other  Program  Issues 

The  Navy  is  seeking  authority  from  Congress  to  use 
what  it  calls  a  "block  buy  contract"  for  SSC  craft  in 
fiscal  years  2017  and  2018.  The  program  released 
a  draft  solicitation  for  these  fiscal  years  in  November 
2016  and  plans  to  release  a  final  solicitation  in 
February  2017.  Because  the  program  does  not  yet 
have  authority  for  a  block  buy,  the  draft  solicitation 
sought  pricing  for  both  block  buy  and  annual 
procurement  contracting  methods.  Program  officials 
estimate  the  block  buy  would  reduce  costs  by  3 
percent  compared  to  an  annual  procurement. 
However,  the  unknowns  that  persist  in 
developmental  testing  undermine  the  Navy's 
business  case  for  the  block  buy  strategy.  Until  this 
testing  is  completed — and  the  program 
demonstrates  the  craft  under  construction  are  of  a 
configuration  that  meets  requirements — program 
execution  is  likely  to  be  hampered,  regardless  of  the 
contracting  method. 

Program  Office  Comments 

When  provided  with  a  draft  of  this  assessment, 
program  officials  noted  the  following.  The  program 
is  currently  in  the  low-rate  production  phase.  The 
design  remains  mature,  and  potential  issues  are 
being  identified  and  addressed  through  integration 
and  test.  Currently,  four  craft  are  under  construction 
with  a  fifth  craft  in  the  prefabrication  stage.  In 
addition  to  the  items  discussed  in  the  Technology 
and  Design  Maturity  section,  after  delays  due  to  a 
fire  at  the  manufacturer,  propellers  for  the  test  and 
training  craft  are  scheduled  to  be  delivered  in 
February  2017.  The  program  is  focused  on 
delivering  this  craft  in  2017  and  beginning  post¬ 
delivery  testing  in  fiscal  year  2018. 
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Lead  Component:  Navy 


Common  Name:  VH-92A 


VH-92A  Presidential  Helicopter  Replacement  Program 

The  Navy's  VH-92A  program  provides  new 
helicopters  for  executive  transport  of  the  President, 

Vice  President,  heads  of  state,  and  others.  A 
successor  to  the  VH-71  program,  which  the  Navy 
canceled  due  to  cost  growth,  schedule  delays,  and 
performance  shortfalls,  the  VH-92A  will  replace  both 
legacy  VH-3D  and  VH-60N  aircraft.  The  VH-92A  is 
expected  to  provide  improved  performance  and 
communications  capabilities,  while  offering 
increased  passenger  capacity. 


Source:  ©Sikorsky  Aircraft  Corporation,  a  Lockheed  Martin  Company. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  in  millions) 

Prime  contractor:  Sikorsky  Aircraft 

As  of 

Latest 

Percent 

Corp. 

04/2014 

08/2016 

change 

Program  office:  Patuxent  River,  MD 

Research  and  development  cost 

$2,714.4 

$2,522.2 

-7.1% 

Funding  needed  to  complete: 

Procurement  cost 

$2,128.0 

$2,296.6 

7.9% 

R&D:  $1 ,343.0  million 

Total  program  cost 

$4,842.4 

$4,818.9 

-0.5% 

Procurement:  $2,296.6  million 

Program  unit  cost 

$210,540 

$209,516 

-0.5% 

Total  funding:  $3,639.6  million 

Total  quantities 

23 

23 

0.0% 

Procurement  quantity:  17 

Acquisition  cycle  time  (months) 

76 

75 

-1.3% 

The  VH-92A  program  completed  its  system-level 
critical  design  review  (CDR)  in  July  2016  with 
stable  designs  for  the  VH-92A  aircraft  and  all  but 
one  subsystem,  which  the  contractor  had  not  yet 
incorporated  into  its  design.  The  program  does 
not  identify  any  critical  technologies.  The  program 
is  designing  one  subsystem,  the  mission 
communications  system,  which  employs 
components  currently  in  use  on  other  platforms. 
The  contractor  identified  design  deficiencies  with 
the  aircraft's  front  and  rear  doors.  In  response,  the 
contractor  has  undertaken  efforts  to  address 
these  deficiencies  and  ensure  timely  delivery  of 
the  development  models.  The  program  plans  to 
enter  production  in  2019  prior  to  completing 
developmental  testing — an  approach  inconsistent 
with  best  practices. 
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Attainment  of  Product  Knowledge 


As  of  January  2017 


Resources  and  requirements  match 


Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

Complete  a  system-level  preliminary  design  review 


Product  design  is  stable 


Release  at  least  90  percent  of  design  drawings 
Test  a  system-level  integrated  prototype 


STATUS  CURRENT 
AT  STATUS 


DEVELOPMENT 

START 


O 


DESIGN  REVIEW 


O  O 


Manufacturing  processes  are  mature 


Demonstrate  Manufacturing  Readiness  Level  of  at  least 
9  or  critical  processes  are  in  statistical  control 

Demonstrate  critical  processes  on  a  pilot  production  line 
Test  a  production-representative  prototype 


PRODUCTION 

START 


^  Knowledge  attained 
o  Knowledge  not  attained 


Information  not  available 
Not  applicable 
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Common  Name:  VH-92A 


VH-92A  Program 

Technology  Maturity 

The  VH-92A's  acquisition  strategy  is  underpinned 
by  use  of  a  commercial  aircraft  and  mature 
technologies  to  reduce  the  potential  for  problems 
during  development  and  early  testing.  As  such,  the 
program  does  not  identify  any  critical  technologies. 
The  government  designed  mission  communications 
system  (MCS)  is  the  only  program  technology  that 
is  not  in  use  in  another  system  in  the  same 
configuration  required  for  the  VH-92A.  First  flight 
with  a  fully  configured  MCS  is  planned  for  June 
2017.  The  delays  in  the  aircraft’s  build  schedule 
caused  by  design  revisions  and  availability  of  parts 
caused  a  2-month  delay  in  first  flight.  The  MCS 
consists  of  both  contractor  and  government 
furnished  equipment  either  commercially  available 
or  already  in  use  on  other  systems.  According  to  the 
program  office,  the  MCS  software  build  is 
proceeding  as  scheduled,  and  updates  are  planned 
for  May  and  October  2017,  including  those  to 
address  known  software  deficiencies  associated 
with  bandwidth  and  compatibility  issues  with  ground 
network  systems.  The  VFI-92A  program  is  pursuing 
approaches  including  having  the  MCS  software  be 
"user  modifiable  software,"  which  will  allow  the 
program  to  modify  the  software  and  maintain 
compliance  with  the  Federal  Aviation  Administration 
certification. 

Design  Maturity 

According  to  the  program  office,  the  contractor 
released  over  90  percent  of  VFI-92A's  design 
drawings  at  the  time  of  the  July  2016  CDR.  These 
drawings  represent  the  modifications  to  the  baseline 
airframe — Sikorsky's  S-92A  aircraft.  In  addition,  the 
program  assessed  designs  for  nearly  all  key  sub¬ 
systems  and  components  as  stable  at  CDR.  The 
only  exception  was  the  Wide-band  Line-of-Sight 
communication  system,  which  the  contractor  had 
not  yet  incorporated  into  its  design.  The  program 
has  experienced  a  few  design  challenges 
associated  with  the  airframe  and  integration.  For 
example,  the  contractor  has  undertaken  extensive 
design  and  structural  analyses  and  studies  for  rear 
and  forward  air-stair  efforts  to  ensure  the  VH-92A's 
design  does  not  affect  existing  aircraft  loads,  which 
could  impact  its  airworthiness  certification  and  to 
ensure  timely  delivery  of  the  development  models. 
The  MCS/CSO  (Communication  System  Operator) 
interface  design  has  also  progressed  slower  than 


planned;  however,  the  program  did  not  experience 
any  delays  to  key  milestones.  Currently,  the 
contractor  is  prioritizing  work,  increasing  resources 
to  address  these  challenges,  and  is  identifying 
areas  later  in  the  aircraft  build  process  where  cost 
efficiencies  can  be  applied. 

Production  Maturity 

The  program  is  planning  to  enter  low-rate  initial 
production  in  January  2019,  prior  to  completing 
system-level  developmental  testing  in  September 
2019 — an  approach  inconsistent  with  best 
practices.  The  program  intends  to  evaluate  VH- 
92A's  readiness  to  enter  production  following  an 
operational  assessment  in  fiscal  year  2018  and 
plans  to  demonstrate  critical  manufacturing 
processes  on  a  pilot  production  line  prior  to  entering 
low-rate  initial  production. 

Other  Program  Issues 

The  Navy  determined  it  no  longer  required  one 
capability  planned  for  VFI-92A  and  subsequently 
executed  a  contract  modification.  According  to 
program  officials,  as  part  of  the  related  negotiation, 
the  government  and  contractor  agreed  on  an 
appropriate  credit  amount  in  July  2016,  which  the 
Navy  will  apply  to  other  program  areas. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  technical  comments,  which 
were  incorporated  where  deemed  appropriate. 
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Common  Name:  LX(R) 

Amphibious  Ship  Replacement  (LX(R)) 


The  Navy’s  LX(R)  program  plans  to  build  a  new  class  of 
ships  to  replace  existing  amphibious  ships.  The  primary 
function  of  these  ships  is  to  transport  Marines  and  their 
equipment  to  distant  operating  areas  and  enable 
expeditionary  operations  ashore.  The  LX(R)  will  include  a 
larger  hull  than  the  retiring  ships,  and  can  also  be  used 
for  non-combat  operations  due  to  its  storage  space  and 
ability  to  transfer  people  and  supplies.  Starting  in  fiscal 
year  2020,  the  Navy  plans  to  procure  1 1  ships  with 
delivery  of  the  first  LX(R)  in  2025. 


Source:  Computer  Science  Corp  (CSC)  pursuant  to  LX(R)  Design 
Support  Contract.  Notional  Concept  LX(R). 


Current  Status 

The  LX(R)  design  is  based  on  the  San  Antonio  (LPD  17)  class  amphibious  ships,  which  the  Navy  believes  will 
reduce  the  cost  risks  associated  with  a  new  design  by  leveraging  existing  design  stability  as  well  as  not  adding 
any  new  critical  technologies.  However,  the  existing  LPD  17  design  is  unaffordable  for  the  LX(R)  program,  so 
the  Navy  is  aiming  to  reduce  costs  while  ensuring  that  the  LX(R)  meets  the  requirements  validated  by  the  Joint 
Staff  in  October  2016.  The  Navy  used  a  limited  competition  approach  that  combined  LX(R)  design  efforts  with 
acquisition  activities  for  the  next  America  Class  Amphibious  Assault  Ship  (LHA  8)  and  the  first  six  John  Lewis 
Class  Fleet  Replenishment  Oilers  (T-AO  205).  In  2015,  the  Navy  issued  a  combined  solicitation  that  was 
limited  to  two  contractors — Huntington  Ingalls  Industries  and  General  Dynamics  NASSCO — for  detail  design 
and  construction  of  T-AO  205  ships  and  LHA  8,  as  well  as  early  design  work  for  LX(R).  In  201 6,  Huntington 
Ingalls  was  awarded  75  percent  of  the  LX(R)  contract  design  hours,  and  General  Dynamics  the  remainder. 

Also  in  2016,  Congress  appropriated  $250  million  in  advanced  procurement — three  years  earlier  than  the  Navy 
had  programmed — to  accelerate  LX(R).  Program  officials  said  they  will  use  those  funds  for  long  lead  items  like 
engines,  which  could  position  the  Navy  to  begin  construction  early.  Although  the  Navy  still  plans  to  award  the 
construction  contract  in  fiscal  year  2020,  it  now  plans  to  start  lead  ship  construction  in  fiscal  year  2021  instead 
of  fiscal  year  2022,  which  should  accelerate  delivery  from  fiscal  year  2026  to  fiscal  year  2025.  Despite  this 
acceleration,  the  Navy  has  not  finalized  its  acquisition  strategy,  but  expects  to  do  so  in  fiscal  year  2017. 

Further,  DOD  delegated  program  oversight  to  the  Navy,  which  currently  anticipates  LX(R)  will  enter  the 
acquisition  process  at  production  in  fiscal  year  2018. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2017  dollars): 

Total  program:  $1 ,886.2  billion  (fiscal  years  201 0-2021 ) 

Research  and  development:  $1 72.2  million  (fiscal  years  201 0-2021 ) 

Procurement:  $1 ,714  billion  (fiscal  years  2016-2021) 

Quantity:  1 1  (only  one  ship  will  be  acquired  between  fiscal  years  201 6-2021 ) 

Next  Major  Program  Event:  Release  of  request  for  proposal  for  long  lead  time  materials  and  detail  design 
contract  award,  fourth  quarter  fiscal  year  2017 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  program  office  provided 
technical  comments,  which  were  incorporated  where  deemed  appropriate. 
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Common  Name:  DDG  51  Flight  III 

DDG  51  Flight  III  Destroyer  (DDG  51  Flight  III) 


The  DDG  51  Flight  III  destroyer  will  be  a  multi- 
mission  ship  designed  to  operate  against  air, 
surface,  and  subsurface  threats.  Compared  to  existing 
Flight  IIA  ships,  Flight  III  ships  will  provide  increased 
ballistic  missile  and  area  air  defense  capabilities  to 
the  fleet.  Flight  Ill’s  planned  configuration  changes 
include  replacing  the  current  SPY-1  D(V)  radar  with 
the  Air  and  Missile  Defense  Radar  program’s  SPY-6 
radar.  The  Navy  plans  to  acquire  a  total  of  22  Flight  III 
ships. 


Source:  Huntington  Ingalls  Industries,  Pascagoula,  MS. 


Current  Status 

The  Navy  continues  Flight  III  detail  design  activities,  which  include  extensive  changes  to  the  ship’s  hull, 
mechanical,  and  electrical  systems  to  incorporate  the  SPY-6  radar  and  restore  safety  margins  to  the  weight 
and  stability  limitations  of  the  ship.  To  reduce  technical  risk,  the  Flight  III  design  includes  new  electrical  and 
air  conditioning  systems  that  are  currently  in  use  on  other  ship  classes.  The  existing  DDG  51 -class  ship 
design  is  dense  and  creates  challenges  for  Flight  III  design  and  construction,  such  as  having  to  rearrange 
equipment  to  fit  new  items  and  potentially  higher  construction  costs  due  to  inefficiencies  caused  by  working  in 
tight  spaces.  The  Navy  began  Flight  III  zone  design — three-dimensional  modeling  of  the  individual  ship  units 
— in  October  2015  and  plans  to  complete  zone  design  before  starting  construction  in  spring  2018.  The  Navy’s 
plans  are  ambitious,  considering  the  amount  and  complexity  of  the  remaining  design  work.  For  example,  one 
shipbuilder  was  not  scheduled  to  begin  zone  design  on  the  five  zones  requiring  the  most  complex  changes 
until  December  2016,  which  may  provide  insufficient  time  to  discover  and  address  problems. 

The  Navy  planned  to  modify  its  existing  Flight  IIA  multiyear  procurement  contracts  to  construct  the  first  three 
Flight  III  ships.  In  fiscal  year  2016,  the  Navy  received  $1  billion  in  construction  funding  to  procure  an 
additional  ship.  The  Navy  now  plans  to  use  this  funding  to  acquire  the  first  Flight  III  ship  under  a  fixed  price 
incentive  engineering  change  proposal.  The  Navy  is  revising  its  Flight  III  acquisition  strategy,  including  an 
updated  acquisition  program  baseline  and  cost  estimate,  for  an  upcoming  but  unscheduled  program  review 
ahead  of  Flight  III  construction  start. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2017  dollars): 

Total  program:  $43,860.4  million 
Research  and  development:  $4,082.9  million 
Procurement:  $39,777.5  million 
Quantity:  22  (procurement) 

Next  Major  Program  Event:  Award  of  first  Flight  III  construction  contract,  fiscal  year  2017. 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  program  office  provided 
technical  comments,  which  were  incorporated  where  deemed  appropriate.  In  addition,  program  officials 
noted  that  DDG  51  Flight  III  design  is  progressing  to  support  the  start  of  construction  with  a  mature  and 
stable  design.  Officials  also  noted  that  the  Flight  III  Critical  Design  Review  was  successfully  completed  in 
November  2016  and  that  initial  functional  design  and  transition  design  are  complete  with  zone  design 
underway  and  expected  to  complete  prior  to  the  start  of  construction. 
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Common  Name:  T-AO  205 


John  Lewis  Class  Fleet  Replenishment  Oiler  (T-AO  205)  [formerly  T-AO(X)] 


The  T-AO  205  program  is  intended  to  replace  the  Navy’s 
15  existing  T-AO  187  Class  Fleet  Oilers,  which  are 
nearing  the  end  of  their  service  lives.  The  primary 
mission  T-AO  205  oilers  will  fulfill  is  replenishment  of  bulk 
petroleum  products,  dry  stores  and  packaged  cargo,  fleet 
freight,  mail,  and  personnel  to  other  vessels  while 
underway.  The  Navy  plans  to  procure  a  total  of  17  ships, 
starting  with  the  first  in  fiscal  year  2016  and  the 
remaining  ships  at  a  rate  of  one  per  year  beginning  in 
fiscal  year  2018. 

Source:  U.S.  Navy.  Notional  Concept  T-AO(X) 


Current  Status 

The  Navy  assessed  the  program’s  three  critical  technologies — all  associated  with  a  new  underway 
replenishment  system  for  transferring  cargo  to  other  ships — as  fully  mature  based  on  the  results  of  land-based 
and  at-sea  prototype  testing.  Each  T-AO  205  oiler  will  have  greater  cargo  capacity  than  legacy  T-AO  187 
class  oilers  to  replenish  items  such  as  food  and  spare  parts. 

In  June  2016,  the  Navy  awarded  a  detail  design  and  construction  (DD&C)  contract  after  limited  competition 
that  combined  the  purchases  of  the  first  6  T-AO  205  class  ships  with  LHA  8  and  initial  engineering  efforts  for 
LX(R),  which  are  amphibious  ships.  The  Assistant  Secretary  of  the  Navy  for  Research,  Development,  and 
Acquisition — the  T-AO  205  program’s  milestone  decision  authority — approved  a  tailored  acquisition  approach 
wherein  the  program  awarded  a  lead  ship  DD&C  contract  before  conducting  a  preliminary  design  review 
(PDR).  According  to  program  officials,  the  program  plans  to  conduct  the  PDR  1 2  months  after  award  of  the 
DD&C  contract  because,  when  adapting  a  commercial  ship  design  for  Navy  use,  some  information  typically 
evaluated  in  a  PDR  can  only  be  determined  after  developing  an  allocated  baseline,  which  is  essentially 
defining  all  subsystems  and  how  they  will  work  together  in  the  overall  ship  design. 

Congress  authorized  the  Navy  to  award  what  the  Navy  refers  to  as  a  block  buy  contract  covering  the  second 
through  sixth  ships  of  the  class.  The  remaining  1 1  ships  of  the  class,  according  to  Navy  officials,  will  be 
acquired  at  a  rate  of  one  ship  per  year  under  two  additional,  possibly  firm-fixed-price,  contracts.  In  addition, 
program  officials  said  that  they  are  working  with  the  prime  contractor  to  reduce  costs  and  have  incentivized 
cost  reduction  in  the  lead  ship  DD&C  contract.  The  program’s  estimated  acquisition  costs  beyond  fiscal  year 
2021  remain  in  development. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2017  dollars): 

Total  program:  $2,357.78  million  (fiscal  years  2011-2021) 

Research  and  development:  $57.78  million  (fiscal  years  2011-2015) 

Procurement:  $2,300.00  million  (fiscal  years  2016-2021) 

Quantity:  5  (fiscal  years  2016-2021) 

Next  Major  Program  Event:  Preliminary  Design  Review,  April  2017 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  program  office  provided 
technical  comments,  which  were  incorporated  where  deemed  appropriate. 
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Common  Name:  MQ-25 


MQ-25  Unmanned  Aircraft  System  (MQ-25) 


The  Navy’s  MQ-25  will  be  a  catapult-launched  unmanned 
aircraft  system  (UAS)  operating  from  aircraft  carriers. 
When  complete,  it  will  provide  a  robust  refueling  capability 
for  the  carrier  air  wing  with  an  intelligence,  surveillance, 
and  reconnaissance  (ISR)  capability.  The  system  is  made 
up  of  an  air  segment,  a  carrier  segment,  and  a  control 
system  and  connectivity  segment.  It  is  the  outcome  of  a 
restructuring  of  the  former  Unmanned  Carrier-Launched 
Airborne  Surveillance  and  Strike  (UCLASS)  program. 


Source.  Jeff  Hobralh,  Chief.  USN  (Ret)  President,  Naval  Tees.  LLC. 


Current  Status 

In  February  2016,  following  a  review  of  its  airborne  ISR  portfolio,  DOD  restructured  the  UCLASS  program  to 
create  the  Carrier  Based  Aerial  Refueling  System,  subsequently  designated  the  MQ-25.  This  decision 
followed  debate  regarding  the  intended  mission  and  required  capabilities  of  UCLASS.  In  particular,  DOD 
evaluated  whether  the  system’s  primary  role  should  be  largely  surveillance  with  limited  strike  capability  or 
largely  strike  with  limited  surveillance  capability.  Ultimately,  DOD  elected  to  pursue  an  aerial  refueling 
capability  with  some  ISR  capability  in  a  newly  restructured  program,  which  the  Navy  included  in  its  fiscal  year 
2017  budget  submission.  According  to  Navy  officials,  the  MQ-25’s  aerial  refueling  capability  will  reduce  the 
need  for  F/A-18E/F  Super  Flornet  aircraft  to  provide  refueling,  freeing  them  for  strike  missions  and  preserving 
service  life. 

The  Navy  plans  to  release  a  request  for  proposals  for  air  system  development  by  summer  2017  and  award  a 
contract  one  year  later.  The  Navy  does  not  plan  to  conduct  a  system-level  preliminary  design  review  before 
starting  development.  Instead,  the  Navy  will  use  knowledge  gained  through  previously  conducted  reviews  with 
the  four  UCLASS  contractors.  Additionally,  in  September  and  October  2016,  the  Navy  awarded  cost-plus- 
fixed-fee  contracts  valued  between  $35.8  million  and  $43.7  million  to  each  of  those  four  contractors  to  conduct 
risk  reduction  activities,  including  concept  refinement  and  requirements  trade  analysis.  While  four  critical 
technologies  were  identified  for  UCLASS,  the  Navy  has  not  yet  identified  any  for  the  MQ-25.  According  to 
Navy  officials,  provisions  will  be  incorporated  for  potential  future  strike  capabilities,  but  requirements  for  initial 
operating  capability — expected  by  the  end  of  fiscal  year  2026 — will  focus  on  refueling  and  ISR  capabilities. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2017  dollars): 

Total  program:  TBD 

Research  and  development  (fiscal  years  2012-2021):  $3,122  million 
Procurement:  TBD 

Military  construction  (fiscal  years  2012-2021):  $39  million 
Quantity:  TBD 

Next  Major  Program  Event:  Release  request  for  proposals  for  engineering,  manufacturing,  and  development 
of  air  system,  summer  2017. 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  program  office  provided 
technical  comments,  which  were  incorporated  where  deemed  appropriate. 
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Navy  Frigate 


Common  Name:  Frigate 


The  Navy’s  new  frigate  is  expected  to  be  a  multi¬ 
mission  small  surface  combatant  derived  from  minor 
modifications  to  a  Littoral  Combat  Ship  (LCS)  design. 
The  frigate  will  incorporate  existing  surface  and 
subsurface  capabilities,  and  add  over-the-horizon 
missile  and  electronic  warfare  capabilities,  among  other 
improvements.  Compared  to  the  existing  LCS,  the 
frigate  is  expected  to  provide  improved  lethality  and 
survivability.  The  Navy  plans  to  acquire  a  total  of  12 
frigates  beginning  in  fiscal  year  2018. 


Source:  Lockheed  Martin  (left);  General  Dynamics  (right). 


Current  Status 

The  Navy  has  not  yet  fully  defined  the  frigate’s  design  and  cost.  Despite  these  uncertainties,  the  Navy’s 
current  acquisition  strategy — approved  in  March  2016 — indicates  it  intends  to  request  authorization  from 
Congress  in  201 7  to  use  what  it  calls  block  buy  contracting  to  buy  all  of  the  planned  frigates  and  for  funding 
the  lead  ship  before  solidifying  realistic  cost  and  design  parameters.  This  acquisition  strategy  includes  the 
Navy  obtaining  block  buy  option  pricing  in  2017  from  both  LCS  shipyards  for  12  LCS.  Then,  the  Navy  plans  to 
combine  frigate-specific  design  upgrades  with  the  LCS  priced  options  to  inform  its  decision  on  a  single  frigate 
contractor  and  design  in  July  2018.  The  estimated  cost  for  the  program  is  uncertain — the  Navy  expects  to 
have  a  formal  estimate  in  May  2017,  and  DOD’s  Office  of  Cost  Assessment  and  Program  Evaluation  has 
indicated  an  independent  cost  estimate  will  be  completed  in  fiscal  year  2018. 

The  Navy  is  currently  reviewing  frigate  build  specifications  received  from  the  two  LCS  shipbuilders  in  the  lead 
up  to  soliciting  proposals  in  September  201 7  for  the  frigate  design  upgrades.  The  Navy  plans  to  review  these 
proposals  and  award  the  frigate  to  a  single  shipyard  before  beginning  detail  design — a  critical  phase  of  design 
that  more  fully  defines  ship  construction  needs  and  cost  expectations.  Although  the  Navy  has  stated  detail 
design  will  be  completed  before  frigate  construction  begins  in  fiscal  year  2020,  awarding  the  contract  for 
frigate  construction  before  beginning  frigate-specific  detail  design  activities  reduces  the  knowledge  that  will  be 
available  to  help  inform  decisions  by  the  shipbuilders  and  the  Navy  in  the  solicitation  and  contract  award 
process. 

Estimated  Program  Funding  and  Quantity  (fiscal  year  2017  dollars): 

Total  Program  (fiscal  years  2016-2025):  $8,317.9  million 
Research  and  development  (fiscal  years  2016-2021):  $312.4  million 
Procurement  (fiscal  years  2018-2025):  $8,005.4  million 
Quantity:  12 

Next  Major  Program  Event:  Design  Maturity  Review,  August  2017 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  program  office  stated  that 
the  frigate  will  provide  significant  improvements  in  lethality  and  survivability.  The  program  office  also  noted 
that,  while  frigate  detail  design  will  not  be  completed  before  contract  award,  it  has  been  completed  for  any 
areas  common  with  LCS,  which  total  greater  than  60  percent  for  both  LCS  variants.  The  program  office 
provided  technical  comments  as  well,  which  were  incorporated  where  deemed  appropriate. 
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P-8A  Increment  3 


Common  Name:  P-8A 


The  Navy’s  P-8A  Increment  3  is  intended  to  provide 
enhanced  capabilities  to  the  P-8A  aircraft  in  four  sets 
of  capability  improvements.  The  first  two  sets  include 
communications,  radar,  and  weapons  upgrades,  which 
will  be  incorporated  into  the  existing  P-8A  architecture. 
The  second  two  sets  will  establish  new  open  systems 
architecture  and  integrate  improvements  to  the  combat 
system’s  ability  to  process  and  display  classified 
information  and  its  search,  detection,  and  targeting 
capabilities. 


Source:  U  S.  Navy. 


Current  Status 

In  March  201 6,  the  Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics  approved  an 
updated  P-8A  acquisition  strategy,  incorporating  Increment  3  capabilities  as  a  series  of  engineering  change 
proposals  to  the  existing  program.  Increment  3  will  no  longer  be  managed  as  a  separate  program  as  previously 
planned.  In  May  2016,  the  Under  Secretary  re-designated  the  P-8A  as  an  acquisition  category  1C  program  and 
delegated  Milestone  Decision  Authority  to  the  Assistant  Secretary  of  the  Navy  for  Research,  Development  and 
Acquisition.  The  acquisition  program  baseline  was  updated  to  support  the  strategy  change  in  June  2016. 

The  P-8A  Increment  3  program’s  objective  is  to  deliver  improved  capabilities  while  introducing  competition  and 
increasing  the  government’s  role  in  developing  future  upgrades.  Program  officials  stated  Increment  3  will 
require  the  integration  of  new  hardware  and  software  based  on  mature  technologies.  As  a  result,  the  program 
has  not  identified  any  critical  technologies.  The  program  office  estimates  this  effort  is  evenly  split  between 
hardware  and  software.  The  Navy  awarded  sole-source  contracts  to  Boeing — the  P-8A  prime  contractor — to 
integrate  capabilities  into  the  existing  aircraft  in  fiscal  years  2018  and  2019.  In  fiscal  years  2022  and  2023,  the 
program  office  plans  to  deliver  an  application  based  open  system  architecture,  which  will  allow  it  to  openly 
compete  the  development  and  integration  of  the  future  capabilities.  This  architecture  upgrade  is  being 
competitively  prototyped  and  executed  as  an  open  source,  collaborative  development.  The  program  office 
awarded  two  software  development  contracts  in  2014.  Program  officials  said  they  will  choose  the  best  parts  of 
each  design  and  integrate  them  into  a  single  architecture.  The  hardware  will  be  funded  as  a  part  of  the 
operating  and  support  costs  of  the  P-8A. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2017  dollars): 

Total  program:  $1,128.91  million 
Research  and  development:  $1 ,128.91  million 
Procurement:  N/A 
Quantity:  109 

Next  Major  Program  Event:  Increment  3  Engineering  Change  Proposal  6  Preliminary  Design  Review,  fourth 
quarter  fiscal  year  2017 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  program  officials  stated  that  the 
P-8A  Increment  3  program  continues  to  execute  the  approved  acquisition  strategy.  This  strategy  includes 
the  engineering  and  contract  work  necessary  to  support  the  efficient  integration  of  required  capabilities,  as 
engineering  change  proposals  into  the  P-8A  baseline. 
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Air  Force  Programs  Summary 


We  completed  individual  assessments  on  18  of  the  Air  Force’s  32  current  and  future  major  defense  acquisition 
programs.  Of  these  18  programs,  14  are,  for  the  most  part,  in  system  development  or  early  production,  while  4  are 
future  programs  that  DOD  expects  to  enter  system  development  in  the  next  few  years.  We  found  the  Air  Force 
currently  estimates  a  need  of  $98.2  billion  in  funding  to  complete  the  acquisition  of  these  18  programs.  In  addition, 
we  compared  these  1 8  programs’  first  full  estimates  of  cost  and  schedule,  as  available,  with  their  current 
estimates  and  found: 

•  net  cost  growth  totals  $39.7  billion,  almost  all  of  which  occurred  in  the  past  5  years  and  is  attributable  to  the 
Evolved  Expendable  Launch  Vehicle  (EELV)  program,  and 

•  program  schedule  delays  average  approximately  3  months. 

Further,  4  of  the  1 8  programs — Enhanced  Polar  System,  EELV,  Small  Diameter  Bomb  Inc  II,  and  Space 
Fence  Inc  I — completed  all  activities  associated  with  the  applicable  knowledge  based  best  practices  we 
assess,  although  these  activities  were  not  fully  complete  at  the  time  the  knowledge  points  were  reached. 


Acquisition  Phase  and  Size  of  the  Currently  Estimated  Acquisition  Cost  for  Summary  of  Knowledge  Attained  to  Date  for 
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o 

o 

SDB  II 

o 

• 

o  • 

•  • 

Space  Fence 
Inc  1 

o 

• 

o  • 

3DELRR 

o 

o 

KP  2 
in  future 

KP  3 
in  future 

•  Cost  growth  of  more  than  15  percent  and/or 
schedule  delays  of  more  than  6  months 

•  Cost  growth  of  1 5  percent  or  less  and  schedule 
delays  of  6  months  or  less 

No  first  full  estimate  available 


Average 
acquisition 
cycle  time 

(in  months) 


85 


3 


•  All  applicable  knowledge  practices 
completed 

OOne  or  more  applicable  knowledge 
practices  were  not  completed 


Knowledge 


practice  is  not  applicable 


Note:  For  acquisition  cycle  time  only  9  programs  were  assessed  m  m  a  Information  not  available  for  knowledge 
Note:  Bubble  size  is  based  on  each  program’s  currently  estimated  as  not  all  programs  contained  sufficient  information  within  their  practice 

future  funding  needed.  first  full  estimates  to  determine  acquisition  cycle  time.  In  addition 

to  research  and  development  and  procurement  costs,  total 
acquisition  cost  includes  acquisition  related  operation  and 
maintenance  and  system-specific  military  construction  costs. 


Source:  GAO  analysis  of  DOD  data.  |  GAO-1 7-333SP 
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Air  Force  Program  Assessments 

2-page  assessments 

Page  number 

B-2  Defensive  Management  System  Modernization  (B-2  DMS-M) 

131 

Combat  Rescue  Helicopter  (CRH) 

133 

Enhanced  Polar  System  (EPS) 

135 

Evolved  Expendable  Launch  Vehicle  (EELV) 

137 

F-15  Eagle  Passive  Active  Warning  Survivability  System  (F-15  EPAWSS) 

139 

F-22  Increment  3.2B  Modernization  (F-22  Inc  3.2B  Mod) 

141 

Family  of  Advanced  Beyond  Line-of-Sight  Terminals  (FAB-T)  Command  Post 
Terminal  (CPT) 

143 

Global  Positioning  System  III  (GPS  III) 

145 

KC-46  Tanker  Modernization  Program  (KC-46A) 

147 

Military  GPS  User  Eguipment  (MGUE)  Increment  1 

149 

Next  Generation  Operational  Control  System  (GPS  OCX) 

151 

Small  Diameter  Bomb  Increment  II  (SDB  II) 

153 

Space  Fence  Ground-Based  Radar  System  Increment  1  (Space  Fence  Inc  1) 

155 

Three-Dimensional  Expeditionary  Long-Range  Radar  (3DELRR) 

157 

1-page  assessments 


Advanced  Pilot  Trainer  (APT) 

159 

Joint  Surveillance  Target  Attack  Radar  System  Recapitalization  (JSTARS  Recap) 

160 

Presidential  Aircraft  Recapitalization  (PAR) 

161 

Weather  Satellite  Follow-on — Microwave  (WSF-M) 

162 
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Lead  Component:  Air  Force 


Common  Name:  B-2  DMS-M 


B-2  Defensive  Management  System  Modernization  (B-2  DMS-M) 


The  Air  Force’s  B-2  DMS-M  program  plans  to 
upgrade  the  aircraft’s  1980s-era  analog  defensive 
management  system  to  a  digital  capability.  This 
system  detects  and  locates  enemy  radar  systems  to 
provide  threat  warning  and  avoidance  information. 
This  upgrade  is  expected  to  improve  the  frequency 
coverage  and  sensitivity  of  the  system,  update  pilot 
displays,  and  enhance  in-flight  rerouting  capabilities 
to  avoid  air  defense  threats.  It  also  expects  to 
improve  the  reliability  and  maintainability  of  the 
DMS  system  and  the  B-2’s  readiness  rate. 


Source:  U.S.  Air  Force. 


▲ 

Program 

start 

(8/11) 


System  development 


Development 

start 

(3/16) 


Production 


Program  Essentials 

Prime  contractor:  Northrop  Grumman 
Program  office:  Wright  Patterson  AFB, 
OH 

Funding  needed  to  complete: 

R&D:  $820.0  million 
Procurement:  $747.5  million 
Total  funding:  $1 ,567.5million 
Procurement  quantity:  16 


▲  ▲ 

GAO  Design 

review  review 

(1/17)  (3/17) 

▲ 

Low-rate 

decision 

(6/20) 

▲ 

Full-rate 

decision 

(10/21) 

Program  Performance  (fiscal  year  2017  dollars  In  millions) 

As  of  Latest  Percent 

05/2016  08/2016  Change 

Research  and  development  cost 

$1,854.7 

$1,854.7 

0.0% 

Procurement  cost 

$747.5 

$747.5 

0.0% 

Total  Program  cost 

$2,602.3 

$2,602.3 

0.0% 

Program  unit  cost 

$130,113 

$130,113 

0.0% 

Total  quantities 

20 

20 

0.0% 

Acquisition  cycle  time  (months) 

124 

130 

4.8% 

The  program  entered  system  development  in 
March  2016  with  four  critical  technologies  nearing 
full  maturity,  but  may  fall  short  of  meeting  two  key 
performance  parameters.  Consistent  with  best 
practices  and  statutory  requirements,  the  program 
completed  a  preliminary  design  review  prior  to 
development  start.  The  program  expected  to 
complete  a  critical  design  review  in  December 
2016,  but  delayed  it  to  study  hardware  changes 
that  may  reduce  costs.  The  program  will  not  test  a 
system  level  integrated  prototype  until  after  this 
review,  and  system  integration  challenges 
remain,  including  a  precise  antenna  installation 
and  calibration  as  well  as  nuclear  hardening  of 
components.  Test  range  availability,  test  data 
turnaround  times,  and  software  development  may 
each  present  challenges  to  meeting  schedules.  In 
addition,  there  are  limitations  to  verifying  the 
system  will  work  as  intended. 


Page  131  GAO-1 7-333SP  Assessments  of  Selected  Weapon  Programs 


Attainment  of  Product  Knowledge 


As  of  January  2017 


Resources  and  requirements  match 


STATUS  CURRENT 
AT  STATUS 


•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

DEVELOPMENT 

START 

• 

• 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

o 

o 

•  Complete  a  system-level  preliminary  design  review 

• 

• 

Product  design  is  stable 

DESIGN  REVIEW 

•  Release  at  least  90  percent  of  design  drawings 

•  Test  a  system-level  integrated  prototype 

Manufacturing  processes  are  mature 

PRODUCTION 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

START 

9  or  critical  processes  are  in  statistical  control 
Demonstrate  critical  processes  on  a  pilot  production  line 
Test  a  production-representative  prototype 


^  Knowledge  attained 
O  Knowledge  not  attained 


Information  not  available 
Not  applicable 


Common  Name:  B-2  DMS-M 


B-2  DMS-M  Program 

Technology  Maturity 

The  program  entered  system  development  in  March 
201 6  with  four  critical  technologies  nearing  maturity. 
A  receiver-processor  for  bands  1-3  has  been  flight 
tested  on  another  program.  When  the  program 
entered  the  technology  development  phase  in  201 1 , 
the  other  3  technologies — band  4  antenna,  band  4 
receiver-processor,  and  geo-location  algorithm — 
were  immature  with  only  limited  demonstrations 
completed.  The  Air  Force  tested  the  fourth 
technology,  a  receiver-processor  for  bands  1-3  on 
another  program.  The  program  office  subsequently 
refined  requirements  and  better  matched  them  with 
the  maturity  of  the  technologies.  As  a  result,  the  Air 
Force  reduced  planned  DMS-M  capabilities,  which 
the  user  determined  was  an  acceptable  trade  given 
the  additional  cost  and  time  needed  to  meet  the 
more  demanding  requirements.  The  program  office 
currently  projects  meeting  two  key  performance 
parameters  related  to  these  technologies  could  be 
at  risk,  but  it  expects  to  mitigate  these  risks  over  the 
next  4  years  prior  to  production  start. 

Design  Maturity 

The  program  expected  to  complete  its  critical  design 
review  in  December  201 6  with  nearly  1 00  percent  of 
the  expected  drawings  released,  but  without  testing 
a  system  level  integrated  prototype.  Flowever,  the 
program  office  recently  delayed  the  review  to  study 
potential  hardware  changes  that  may  reduce  costs. 
Consistent  with  best  practices  and  statutory 
requirements,  the  program  completed  a  preliminary 
design  review  prior  to  beginning  development.  The 
DMS-M  contractor  estimated  over  80  percent  of  the 
software  development  was  complete.  Flowever, 
technical  integration  challenges  remain.  The  B-2 
DMS-M  is  the  largest  and  most  invasive 
modification  since  the  aircraft  was  fielded.  The 
antenna  design  requires  stringent  tolerance  limits 
for  airframe  installation  due  to  stealth  requirements 
and  requires  precise  calibration.  Most  components 
require  nuclear  hardening  to  meet  operating 
requirements.  The  system  also  requires  a  tightly 
coupled  hardware  and  software  integration  to 
enable  threat  assessment  and  auto-rerouting  for 
enhancing  B-2  survivability. 


Other  Program  Issues 

The  total  estimated  cost  to  develop  and  equip  20  B- 
2s  with  the  DMS-M  has  increased  about  $500 
million  from  estimates  made  at  the  start  of 
technology  development  phase  in  2011.  The  Air 
Force  awarded  a  cost  reimbursable  type  contract  for 
system  development  with  incentive  fees  tied  to  cost 
and  performance.  About  two-thirds  of  the  available 
fee  is  targeted  at  incentivizing  the  contractor  to 
complete  system  verification,  software,  and  test 
readiness  events  on  time.  If  the  contractor  is 
successful,  the  program  office  predicts  the 
estimated  program  cost  would  be  reduced  by 
$209.8  million.  Flowever,  the  contractor  has  had 
difficulties  in  meeting  past  schedules.  Challenges 
remain  that  could  impact  its  ability  to  meet  the 
current  schedule,  including  test  range  availability, 
test  data  turnaround  times,  and  timely  completion  of 
software  development. 

Operational  testers  identified  three  potential 
limitations  to  verifying  that  the  DMS-M  works  as 
intended:  (1)  testing  in  a  dense  signal  environment 
will  be  limited  to  modeling  and  simulation;  (2) 
determining  whether  the  system  is  meeting 
reliability  and  availability  requirements  due  to  the 
limited  number  of  flight  test  hours;  and  (3)  testing 
on-aircraft  cybersecurity  may  not  be  practical  due  to 
security  and  test  asset  concerns. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  technical  comments,  which 
were  incorporated  where  deemed  appropriate. 
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Lead  Component:  Air  Force 


Common  Name:  CRH 


Combat  Rescue  Helicopter  (CRH) 

The  Air  Force's  Combat  Rescue  Helicopter  (CRH)  is 
intended  to  recover  personnel  from  hostile  or  denied 
territory  as  well  as  conduct  humanitarian,  civil 
search  and  rescue,  and  disaster  relief.  The  CRH  is 
intended  to  replace  aging  HH-60G  Pave  Hawk 
helicopters  with  a  derivative  of  the  operational  UH- 
60M  helicopter.  Planned  modifications  to  the 
commercial  design  include  a  new  mission  computer 
and  software,  a  higher  capacity  electrical  system, 
larger  capacity  main  fuel  tanks,  armor  for  crew 
protection,  and  situational  awareness 
enhancements. 


Source:  ©2015  Sikorsky  Aircraft  Corporation.  Used  with  permission  for  support  of  CRH. 


System  development  >  Production 


▲ 

▲ 

▲ 

▲ 

▲ 

A 

A 

Development 

GAO 

Design 

Low-rate 

Start 

Initial 

Full-rate 

start 

review 

review 

decision 

operational  test 

capability 

decision 

(6/14) 

(1/17) 

(7/17) 

(10/19) 

(10/20) 

(4/21) 

(8/21) 

Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  in  millions) 

Prime  contractor:  Sikorsky  Aircraft 

As  of 

Latest 

Percent 

Corporation 

06/2014 

08/2016 

change 

Program  office:  Wright  Patterson  AFB, 

Research  and  development  cost 

$2,040.2 

$2,040.8 

0.0% 

OH 

Procurement  cost 

$6,362.3 

$6,614.9 

4.0% 

Funding  needed  to  complete: 

Total  program  cost 

$8,427.1 

$8,682.9 

3.0% 

R&D:  $1 ,480.6  million 

Program  unit  cost 

$75,242 

$77,526 

3.0% 

Procurement:  $6,614.9  million 

Total  quantities 

112 

112 

0.0% 

Total  funding:  $8,122.6  million 

Acquisition  cycle  time  (months) 

75 

82 

9.3% 

Procurement  quantity:  103 

Following  system  development  start  in  2014,  the 
CRH  program  completed  a  preliminary  design 
review  and  technology  readiness  assessment  in 
2016,  which  identified  the  radar  warning  receiver 
technology  as  immature.  Until  this  technology 
matures,  the  program's  plan  to  achieve  design 
stability  by  its  2017  critical  design  review  (CDR) 
remains  at  risk.  Additional  knowledge  gaps  will 
persist  after  CDR  given  the  program's  plans  to 
defer  early  prototype  testing  to  2018  and  finish 
software  development  in  2020  after  production 
begins  in  201 9.  The  program  also  plans  for  critical 
manufacturing  processes  to  be  in  statistical 
control  and  to  test  a  production  representative 
prototype  before  the  start  of  production  in  line  with 
best  practices. 
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Attainment  of  Product  Knowledge 


As  of  January  2017 

STATUS  CURRENT 
AT  STATUS 

1  Resources  and  requirements  match 

•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

DEVELOPMENT 

START 

O  O 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

O  O 

•  Complete  a  system-level  preliminary  design  review 

O  • 

1  Product  design  is  stable 

•  Release  at  least  90  percent  of  design  drawings 

DESIGN  REVIEW 

•  Test  a  system-level  integrated  prototype 

1  Manufacturing  processes  are  mature 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

9  or  critical  processes  are  in  statistical  control 

PRODUCTION 

START 

Demonstrate  critical  processes  on  a  pilot  production  line 
Test  a  production-representative  prototype 


^  Knowledge  attained 
O  Knowledge  not  attained 


Information  not  available 
Not  applicable 
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Common  Name:  CRH 


CRH  Program 

Technology  Maturity 

The  CRH  program  began  system  development  in 
2014  without  identifying  and  assessing  the  maturity 
of  its  critical  technologies.  At  the  program’s 
milestone  review  for  entry  into  system  development, 
the  Under  Secretary  of  Defense  for  Acquisition, 
Technology  and  Logistics  (USD  AT&L)  waived  the 
technology  maturity  requirements,  citing  the 
maturity  of  the  program’s  technologies.  Program 
officials  said  in  2016,  however,  the  program 
completed  an  independent  technology  readiness 
assessment,  which  identified  the  one  critical 
technology,  the  radar  warning  receiver,  was 
immature.  While  an  analog  version  currently 
functions  on  other  military  helicopters,  it  will  require 
additional  technology  maturation  for  use  on  the 
CRH  as  a  digital  system  providing  threat  warnings 
and  situational  awareness. 

Design  and  Production  Maturity 

The  CRH  program  conducted  its  preliminary  design 
review  (PDR)  in  April  2016,  almost  2  years  after 
system  development  started — an  approach 
inconsistent  with  best  practices.  USD  AT&L  waived 
the  statutory  requirement  for  earlier  completion  of 
the  PDR.  Although  program  officials  said  no 
significant  risks  or  issues  were  identified  for  the 
CRH  program,  they  are  tracking  several  moderate 
risks,  including  some  related  to  aircraft  weight. 

As  of  August  201 6,  the  program  had  released  about 
84  percent  of  the  expected  drawings  and  estimated 
90  percent  would  be  released  by  the  July  2017 
CDR.  Achieving  this  90  percent  metric  by  CDR 
would  be  consistent  with  best  practices,  but  design 
stability  remains  at  risk  until  the  radar  warning 
receiver  technology  matures.  In  addition,  the  CRH 
program  does  not  plan  to  demonstrate  an  early 
system  prototype  before  CDR,  which  is  inconsistent 
with  best  practices.  Instead,  the  program  plans  to 
complete  a  prototype  demonstration  about  7 
months  after  CDR  in  February  2018  and  finish 
software  development  in  2020,  after  production  is 
already  underway. 

Program  officials  stated  system  prototype  testing  at 
CDR  is  not  needed  for  CRH  since  the  program  is 
modifying  and  enhancing  an  existing  helicopter,  the 
UH-60M,  by  integrating  mature  subsystems  and 
associated  software  into  this  platform.  However,  the 


integration  of  these  subsystems  and  software  could 
be  challenging  and  present  some  risks  that  must  be 
successfully  mitigated  to  achieve  the  desired 
capability.  In  our  previous  work,  we  found  that  when 
acquisition  programs  demonstrate  technology  and 
design  maturity  at  appropriate  points,  they  typically 
have  better  cost  and  schedule  outcomes.  Program 
officials  stated  they  have  planned  for  a  sufficient 
amount  of  developmental  testing  to  reduce  these 
risks.  Systems  integration  laboratory  testing  will 
begin  in  February  2017,  several  months  before  the 
CDR,  although  this  testing  will  use  partial  system 
configurations. 

The  program  plans  to  make  use  of  several  other 
knowledge-based  practices  prior  to  its  CDR, 
including  identifying  critical  manufacturing 
processes  and  completing  failure  modes  and  effects 
analysis  to  increase  confidence  in  the  stability  of  the 
CRH  design  and  mature  its  production  processes. 
The  program  also  plans  for  critical  manufacturing 
processes  to  be  in  statistical  control  and  to  test  a 
production  representative  prototype  before  the  start 
of  production,  which  is  consistent  with  best 
practices. 

Program  Office  Comments 

Program  officials  acknowledged  software 
development  could  continue  through  the  end  of 
developmental  test  and  evaluation  in  early  2020,  but 
full  capability  is  expected  by  CRH's  initial  test 
readiness  review  in  2018.  Therefore,  the  only 
software  development  planned  after  2018  will  be  to 
address  any  flight  test  anomalies,  and  program 
officials  believe  this  will  reduce  the  possibility  of 
changes  being  needed  during  production  in  2019. 
Testing  to  prove  the  technology  maturity  of  the  radar 
warning  receiver  in  a  relevant  environment  is 
planned  to  occur  prior  to  the  critical  design  review  in 
201 7.  To  reduce  the  risk  of  an  unsuccessful  test,  the 
Air  Force  Research  Laboratory  provided  a  test 
station  to  the  radar  warning  receiver  developer  that 
will  allow  for  up  to  4  months  of  early  risk  reduction 
testing. 
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Lead  Component:  Air  Force 


Common  Name:  EPS 


Enhanced  Polar  System  (EPS) 


The  Air  Force's  Enhanced  Polar  System  (EPS)  will 
provide  protected,  next-generation  extremely  high 
frequency  (EHF)  satellite  communications  in  the 
polar  region.  It  will  replace  the  current  Interim  Polar 
System  and  serve  as  a  polar  adjunct  to  the 
Advanced  EPIF  (AEPIF)  system.  EPS  consists  of 
three  segments:  two  EPIF  payloads  hosted  on 
classified  satellites,  a  Control  and  Planning 
Segment  (CAPS),  and  a  gateway  to  connect 
modified  Navy  Multiband  Terminals  to  other 
communication  systems. 


Aircraft,  ground, 
ship,  sub 
terminals 
supported 


Polar  Coverage 
Area  above  65° 
Lat  north 


Gateway 
Clear  AS* 


Payload  ops 
Schriever  AFB 


Payload  in 
standby  as  It  sets 
and  turned  on  as 
it  rises 


Source:  LinQuest  Corporation. 
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Design 

Payload-1 

GAO 

Payload-2 

End 

Operational 
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start 

review 

on  orbit 

review 

on  orbit 

operational  test 

capability 

(12/07) 

(4/14) 

(7/14) 

(3/15) 

(1/17) 

(9-11/17) 

(2/18) 

(6/18) 

Program  Essentials 

Program  Performance  (fiscal  year  2017  dollars  in  millions) 

Prime  contractor:  Northrop  Grumman 

As  of 

Latest 

Percent 

Program  office:  El  Segundo,  CA 

04/2014 

10/2016 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$1,446.8 

$1,430.4 

-1.1% 

R&D:  $73.8  million 

Procurement  cost 

$0.0 

$0.0 

0.0% 

Procurement:  $0.0  million 

Total  program  cost 

$1,446.8 

$1,430.4 

-1.1% 

Total  funding:  $73.8  million 

Program  unit  cost 

$723,414 

$715,186 

-1.1% 

Procurement  quantity:  0 

Total  quantities 

2 

2 

0.0% 

Acquisition  cycle  time  (months) 

126 

126 

0.0% 

The  EPS  program  entered  development  in  April 

2014.  The  three  segments  of  EPS — payloads, 
CAPS,  and  the  gateway — are  in  various  stages  of 
completion  with  mature  technologies  and  a  stable 
design.  The  EPS  program's  two  EPIF  payloads 
are  built;  the  first  is  onorbit  and  the  second  will  be 
available  for  on-orbit  testing  in  fall  2017.  The 
gateway  site  installation  is  complete  and  testing 
was  completed  in  December  2015.  CAPS 
completed  software  development  in  October 

2015,  but  implementation  of  cybersecurity 
updates  delayed  delivery  of  the  segment  to  April 
2017.  CAPS  delays  have  pushed  the  expected 
completion  of  on-orbit  inter-segment  testing, 
which  will  test  all  three  system  elements  together, 
from  August  2016  to  July  2017.  Operational 
testing  is  scheduled  for  completion  in  2018  and  is 
the  last  significant  event  scheduled,  as  there  are 
no  production  related  decisions  for  the  program. 


Attainment  of  Product  Knowledge 


As  of  January  2017 


Resources  and  requirements  match 


Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

Complete  a  system-level  preliminary  design  review 


Product  design  is  stable 


Release  at  least  90  percent  of  design  drawings 
Test  a  system-level  integrated  prototype 


STATUS  CURRENT 
AT  STATUS 


DEVELOPMENT 

START 


DESIGN  REVIEW 


Manufacturing  processes  are  mature 


Demonstrate  Manufacturing  Readiness  Level  of  at  least 
9  or  critical  processes  are  in  statistical  control 

Demonstrate  critical  processes  on  a  pilot  production  line 
Test  a  production-representative  prototype 


PRODUCTION 

START 


^  Knowledge  attained 
o  Knowledge  not  attained 


Information  not  available 
Not  applicable 
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Common  Name:  EPS 


EPS  Program 

Technology  and  Design  Maturity 

The  EPS  program  formally  entered  development  in 
April  2014  with  all  critical  technologies  for  the 
payload  segment  mature  or  approaching  maturity. 
These  technologies  are  now  fully  mature.  In  July 
2014,  the  program  held  a  system-level  critical 
design  review,  but  the  program  was  unable  to 
provide  the  number  of  engineering  drawings  to  us 
for  our  prior  assessments.  Our  current  assessment 
shows  all  drawings  have  been  released,  indicating  a 
stable  design.  All  three  segments  of  the  program  will 
be  completed  under  a  development  effort,  and  there 
will  be  no  production-related  decisions  for  this 
program. 

Both  payloads  are  built.  The  first  payload  is  on-orbit 
and  the  second  payload  is  integrated  into  the  host 
satellite  and  is  expected  to  be  available  for  on-orbit 
testing  in  fall  2017.  According  to  program  officials, 
all  EPS  hardware  development  and  critical 
technologies  are  associated  with  the  payloads,  but 
the  CAPS  segment  is  the  program’s  critical  path. 

The  only  development  work  remaining  within  the 
program  is  CAPS.  According  to  program  officials, 
CAPS  is  primarily  a  software  development  effort 
and  utilizes  commercial  off-the-shelf  hardware. 
Program  officials  said  CAPS  software  development 
was  completed  in  October  201 5,  but  implementation 
of  cybersecurity  updates  has  required  restructuring 
of  the  CAPS  schedule,  delaying  segment  delivery  to 
April  2017.  DOD  updates  cybersecurity  standards 
quarterly,  which  program  officials  stated  makes  it 
difficult  to  keep  pace  with  ever-increasing  security 
requirements  while  in  software  development. 
According  to  program  officials,  these  requirements 
are  described  in  detail  via  the  Security  Technical 
Implementation  Guidelines  programs  must 
implement,  the  majority  of  which — along  with 
associated  verification — must  occur  manually,  which 
can  be  time  and  labor  intensive. 

The  gateway  segment  primarily  involves  integration 
of  existing  equipment  and  is  considered  low  risk  by 
the  program  office.  Integration  includes  commercial 
off-the-shelf  hardware,  such  as  routing  and 
switching  equipment,  and  terminals  developed 
under  other  programs.  The  Navy's  Space  and  Naval 
Warfare  Systems  Command,  Systems  Center 
Pacific  and  MIT/Lincoln  Laboratory  are  responsible 


for  integrating,  testing,  and  installing  the  gateway 
segment.  Site  installation  for  the  gateway  is 
complete,  and  testing  was  completed  in  December 
2015.  System-level  on-orbit  inter-segment  interface 
testing  of  the  payload,  gateway,  and  CAPS  was 
initially  scheduled  for  completion  in  August  2016, 
but  the  updated  CAPS  schedule  has  delayed 
completion  until  the  end  of  July  2017. 

Other  Program  Issues 

In  our  prior  assessment,  we  reported  initial 
operational  capability  was  delayed  from  fiscal  year 
201 6  to  2018 — a  timeline  that  now  includes  meeting 
both  initial  and  full  operational  capability  milestones. 
The  program  office  considers  implementation  of 
cybersecurity  updates  to  be  low  risk  in  terms  of 
technical  success,  but  medium  risk  in  terms  of 
schedule  margin  for  system-level  testing,  which 
could  further  delay  operational  capability  since  the 
updates  are  changing  critical  path  activities  leading 
up  to  on-orbit  testing. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  technical  comments,  which 
were  incorporated  where  deemed  appropriate. 
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Lead  Component:  Air  Force 


Common  Name:  EELV 


Evolved  Expendable  Launch  Vehicle  (EELV) 

EELV  provides  spacelift  support  for  DOD,  national 
security,  and  other  government  missions.  Currently, 
the  United  Launch  Alliance  (ULA)  and  Space 
Exploration  Technologies  Corporation  (SpaceX)  are 
the  only  certified  providers  of  EELV  launch  services. 

SpaceX’s  Falcon  9  was  certified  to  compete  for 
national  security  launches  in  May  2015  and  won  a 
GPS  III  launch  services  contract  in  April  2016.  ULA 
provides  launch  services  for  EELV  using  two 
families  of  launch  vehicles,  Atlas  V  and  Delta  IV.  We 
assessed  ULA 's  and  SpaceX’s  vehicle  variants. 


Source:  United  Launch  Alliance  and  SpaceX. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  in  millions) 

Prime  contractor:  United  Launch 

As  of 

Latest 

Percent 

Services,  LLC 

10/1998 

12/2015 

change 

Program  office:  El  Segundo,  CA 

Research  and  development  cost 

$1,931.3 

$4,170.4 

115.9% 

Funding  needed  to  complete: 

Procurement  cost 

$16,917.1 

$55,467.3 

227.9% 

R&D:  $1,181.6  million 

Total  program  cost 

$18,848.4 

$59,637.7 

216.4% 

Procurement:  $30,532.8  million 

Program  unit  cost 

$104,135 

$370,421 

255.7% 

Total  funding:  $31,714.3  million 

Total  quantities 

181 

161 

-11.0% 

Procurement  quantity:  77 

Acquisition  cycle  time  (months) 

NA 

114 

NA 

We  assessed  the  EELV’s  technology  and  found  it 
to  be  mature  with  102  successful  launches  as  of 
January  15,  2017.  EELV’s  design  and  production 
maturity  were  not  assessed  using  our  best 
practices,  but  instead  using  an  Aerospace 
Corporation  measure  developed  for  the  program. 
Using  this  measure,  1 2  of  the  15  EELV  launch 
vehicle  variants  offered  by  ULA  demonstrated 
design  maturity,  as  has  SpaceX’s  Falcon  9.  EELV 
is  currently  assessing  options  for  an  alternative  to 
the  Russian-made  RD-180  engine  used  on  the 
Atlas  V  vehicle,  and  is  investing  in  technology 
development  with  industry  through  agreements 
utilizing  DOD’s  "other  transaction"  authorities. 
ULA  phased  out  the  Delta  IV  intermediate  launch 
vehicle,  which  ULA  plans  to  discontinue  by  fiscal 
year  201 8  due  to  its  high  price. 
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Attainment  of  Product  Knowledge 

As  of  January  2017 

STATUS 

AT 

CURRENT  I 
STATUS  1 

I  Resources  and  requirements  match 

DEVELOPMENT 

•  Demonstrate  all  critical  technologies  are  very  close  to 

START 

• 

final  form,  fit,  and  function  within  a  relevant  environment 

"""" 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

.... 

• 

•  Complete  a  system-level  preliminary  design  review 

• 

• 

1  Product  design  is  stable 

DESIGN  REVIEW 

•  Release  at  least  90  percent  of  design  drawings 

.... 

.... 

•  Test  a  system-level  integrated  prototype 

.... 

.... 

1  Manufacturing  processes  are  mature 

PRODUCTION 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

START 

9  or  critical  processes  are  in  statistical  control 

.... 

.... 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

.... 

.... 

•  Test  a  production-representative  prototype 

.... 

.... 

0  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

O  Knowledge  not  attained  Not  applicable 
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Common  Name:  EELV 


EELV  Program 

Technology,  Design,  and  Production  Maturity 

Fourteen  of  15  EELV  variants  offered  by  ULA,  and 
the  Falcon  9  from  SpaceX,  have  flown  at  least  once, 
demonstrating  technology  maturity.  For  design 
stability  and  production  maturity,  launch  vehicles  are 
assessed  using  the  "3/7  reliability  rule"  developed 
by  the  Aerospace  Corporation.  Once  a  variant  is 
launched  successfully  three  times,  its  design  can  be 
considered  stable  and  mature.  Similarly,  if  a  variant 
is  successfully  launched  seven  times,  both  the 
design  and  production  process  can  be  considered 
mature.  Twelve  of  the  ULA  variants  have  achieved 
design  stability,  and  4  have  reached  both  design 
stability  and  production  maturity,  although  some 
variants  are  used  infrequently  and  may  never  reach 
design  stability  or  production  maturity.  The  Falcon  9 
vl  .1  achieved  both  design  stability  and  production 
maturity,  and  a  new  variant — the  Falcon  9  Upgrade, 
which  SpaceX  intends  to  use  going  forward  for 
EELV  launch  service  competitions — first  flew  in 
December  2015  and  was  certified  for  EELV 
launches  in  January  2016.  New  variants  introduce 
changes  to  the  original  design  which,  until  proven 
through  multiple  successful  flights,  pose  potential 
cost  and  schedule  risks. 

Other  Program  Issues 

The  National  Defense  Authorization  Act  (NDAA)  for 
Fiscal  Year  (FY)  2015  prohibited,  with  certain 
exceptions,  the  award  or  renewal  of  a  contract  for 
the  procurement  of  property  or  services  for  space 
launch  activities  under  the  EELV  program  if  such 
contract  carries  out  such  activities  using  rocket 
engines  designed  or  manufactured  in  the  Russian 
Federation.  The  NDAAs  for  fiscal  years  2016  and 
2017  amended  the  exceptions  to  the  prohibition.  In 
early  2016,  the  Air  Force  awarded  agreements, 
utilizing  DOD's  "other  transaction"  authorities,  for 
rocket  propulsion  systems. 

ULA  is  phasing  out  its  Delta  IV  Intermediate  class  of 
vehicles  to  make  way  for  its  new  Vulcan  launch 
vehicle  currently  in  development  with  an  initial 
capability  expected,  according  to  industry-stated 
timelines,  in  2019.  According  to  the  program  office, 
because  the  NDAA  for  Fiscal  Year  2017  further 
amended  the  exceptions  to  the  limitations  in  the 
NDAA  for  Fiscal  Year  201 5  to  allow  certain 
contracts  to  include  the  use  of  a  total  of  18  rocket 
engines  designed  or  manufactured  in  Russia,  the 


Air  Force  may  be  able  to  avoid  a  gap  in  launch 
capability  for  national  security  space  launch 
missions.  However,  Air  Force  officials  stated  they 
consider  these  industry-stated  rocket  propulsion 
system  timelines  aggressive  and  optimistic. 

The  Air  Force  intends  to  procure  three  Delta  IV 
Heavy-launch  vehicles  to  support  near-term 
national  security  launch  requirements.  However, 
while  ULA  has  enough  components  to  support 
these  missions,  if  additional  missions  are  required, 
some  components  are  not  available  from  suppliers 
and  will  require  new  ones.  The  use  of  components 
from  new  suppliers  could  require  substantial  testing, 
certification,  and  additional  cost.  According  to  the 
Air  Force,  the  initial  demonstration  launch  for 
SpaceX's  Falcon  Heavy  is  likely  to  take  place  in 
2017. 

The  Air  Force  is  developing  an  acquisition  approach 
which  supports  two  launch  providers  that  can 
perform  national  security  launches  and  also 
compete  for  commercial  and  civil  government 
launches.  Implementing  this  strategy  may  prove 
challenging  as  the  demand  for  national  security 
launches  is  expected  to  decline  and  providers  will 
have  to  rely  more  heavily  on  civil  government  and 
commercial  launches.  Additionally,  national 
security-unique  requirements,  such  as  additional  lift 
capability,  larger  satellites,  and  increased  structural 
capability  for  heavy  payloads,  can  drive  up  costs.  At 
the  same  time,  commercial  satellites  do  not  require 
the  same  national  security  launch  capability. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  technical  comments,  which 
were  incorporated  where  deemed  appropriate. 
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Lead  Component:  Air  Force 


Common  Name:  F-15  EPAWSS 


F-15  Eagle  Passive/ Active  Warning  and  Survivablity  System  (F-15  EPAWSS) 

The  Air  Force's  F-15  EPAWSS  program  is 
developing  a  new  electronic  warfare  system  for 
fielded  F-15C/E  aircraft.  The  design  of  this 
upgraded  system  leverages  components  currently 
in  use  on  other  military  aircraft  that  will  enhance  the 
F-15's  ability  to  identify  and  neutralize  advanced  air 
and  ground  threat  systems.  Using  an  incremental 
acquisition  approach,  the  Air  Force  will  replace  the 
aircraft’s  legacy  system  with  EPAWSS  in  Increment 
1  and  add  a  new  towed  decoy  and  associated 
countermeasures  in  Increment  2.  We  assessed 
Increment  1 . 


Source:  U.S.  Air  Force. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  in  millions) 

Prime  contractor:  Boeing 

As  of 

Latest 

Percent 

Program  office:  Wright  Patterson  AFB, 

08/2016 

change 

OH 

Research  and  development  cost 

NA 

$1,028.2 

NA 

Funding  needed  to  complete: 

Procurement  cost 

NA 

$4,228.7 

NA 

R&D:  $786.7  million 

Total  program  cost 

NA 

$5,256.9 

NA 

Procurement:  $4,228.7  million 

Program  unit  cost 

NA 

$12,729 

NA 

Total  funding:  $5,015.4  million 

Total  quantities 

NA 

413 

NA 

Procurement  quantity:  403 

Acquisition  cycle  time  (months) 

NA 

84 

NA 

F-15  EPAWSS  entered  system  development  in 
November  2016  with  its  critical  technologies 
approaching  full  maturity.  The  program  did  not 
conduct  competitive  prototyping,  but  consistent 
with  best  practices  and  statutory  requirements,  it 
did  hold  a  preliminary  design  review  before 
development  start  that  validated  the  use  of 
components  from  other  military  aircraft  with  many 
of  them  requiring  little  change.  At  the  March  2017 
critical  design  review,  the  program  will  focus  on 
the  components  that  require  some  hardware  and 
software  development.  The  program  plans  to 
attain  the  recommended  level  of  manufacturing 
readiness,  complete  pilot  production  line 
demonstrations,  and  complete  an  operational 
assessment  with  production  representative 
hardware  all  before  production  start  in  2019,  but 
some  developmental  testing  and  software  design 
work  remains  to  be  completed  after  this  point. 
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Attainment  of  Product  Knowledge 


As  of  January  2017 


Resources  and  requirements  match 


STATUS  CURRENT 
AT  STATUS 


•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

DEVELOPMENT 

START 

• 

• 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

o 

o 

•  Complete  a  system-level  preliminary  design  review 

• 

• 

Product  design  is  stable 

•  Release  at  least  90  percent  of  design  drawings 

DESIGN  REVIEW 

•  Test  a  system-level  integrated  prototype 

Manufacturing  processes  are  mature 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

PRODUCTION 

START 

9  or  critical  processes  are  in  statistical  control 
Demonstrate  critical  processes  on  a  pilot  production  line 
Test  a  production-representative  prototype 


^  Knowledge  attained 
O  Knowledge  not  attained 


Information  not  available 
Not  applicable 


Common  Name:  F-15  EPAWSS 


F-15  EPAWSS  Program 

Technology  and  Design  Maturity 

F-15  EPAWSS  entered  system  development  in 
November  2016  after  testing  by  an  independent 
review  team  demonstrated  its  four  critical 
technologies  as  approaching  full  maturity  in  an 
operationally  relevant,  high-density  radio  frequency 
signal  environment.  Although  best  practices 
recommend  fully  maturing  critical  technologies  to 
final  form,  fit,  and  function  and  demonstrating  them 
in  a  realistic  environment  by  development  start,  the 
program  does  not  plan  to  attain  this  until  2018, 
leaving  it  at  risk  for  flight  test  delays  if  the 
technologies  do  not  function  as  planned.  In  2015, 
the  prime  contractor  for  EPAWSS  system 
integration,  Boeing,  competitively  selected  BAE 
Systems  to  develop  the  electronic  warfare  system, 
but  competitive  prototyping  of  an  EPAWSS  design 
was  not  conducted  due  to  the  leveraging  of  non- 
developmental  components  by  the  offerors,  which 
DOD  determined  minimized  the  potential  benefits  of 
early  system  prototyping. 

Consistent  with  best  practices  and  statute,  the 
program  held  a  preliminary  design  review  with 
Boeing  before  system  development  as  well  as  other 
engineering  reviews  with  Boeing  and  key  sub¬ 
contractors.  These  reviews  validated  the  program's 
strategy  to  leverage  components  from  other  military 
aircraft  in  a  new  design.  Increment  1  consists  of 
three  key  subsystems  that  will  be  replaced  on  the  F- 
15 — the  radar  warning  receiver,  electronic 
countermeasure  processer,  and  countermeasure 
dispenser  system.  The  program  tracks  design 
drawing  estimates  for  both  the  EPAWSS  hardware 
and  aircraft  changes  needed  to  replace  the  legacy 
system  with  EPAWSS.  According  to  program 
officials,  a  significant  portion  of  the  EPAWSS  design 
is  stable  with  many  of  the  components  requiring  little 
or  no  change  from  a  similar  system  already  fielded 
on  a  foreign  military  F-15.  Program  officials  stated 
the  March  2017  critical  design  review  will  focus  on 
the  integration  of  updated  subsystem  components 
that  require  some  hardware  and  software  design 
work. 

Production  Maturity 

Before  production  begins,  the  program  plans  to 
reach  the  level  of  manufacturing  readiness  for 
production  start  recommended  by  best  practices 
and  demonstrate  critical  manufacturing  processes 


on  a  pilot  production  line  to  provide  assurance 
quality  requirements  will  be  met.  The  program  also 
plans  to  complete  ground  testing  and  a  flight  test 
based  operational  assessment  of  production 
representative  hardware  to  ensure  the  system 
performs  as  planned  before  making  a  production 
start  decision  in  2019.  Plowever,  some  software 
development  will  take  place  after  this  point  with 
about  half  of  the  developmental  flight  testing  to  be 
completed  during  production,  leaving  some  risk 
EPAWSS  might  require  unforeseen  design  or 
software  changes  due  to  performance  shortfalls 
discovered  late  in  testing. 

Other  Program  Issues 

To  date,  the  Air  Force  has  only  funded  Increment  1 
of  the  program,  but  currently  plans  to  request  funds 
for  Increment  2  development  beginning  in  fiscal 
year  2018.  Air  Force  officials  assessed  the  new 
Increment  2  hardware  and  technology  as  mature, 
but  it  is  not  yet  qualified  for  use  on  the  F-15.  In  the 
event  that  the  Air  Force  does  not  receive  funding  for 
Increment  2  development  and  production,  program 
officials  stated  that  the  Increment  1  upgrade  alone  is 
a  viable  improvement  to  the  F-15,  as  the  legacy 
system  it  replaces  is  becoming  functionally  obsolete 
and  unsustainable. 

Program  Office  Comments 

According  to  Air  Force  officials,  the  program  is  on- 
track  in  implementing  a  strategy  to  leverage  mature 
non-developmental  components  as  a  key  element 
of  the  program's  efforts  to  drive  positive  schedule, 
affordability,  and  risk  outcomes.  They  also  noted  the 
program  successfully  demonstrated  the  critical 
technologies  needed  to  perform  against  advanced 
threats  in  a  high  fidelity,  operationally  relevant 
environment.  Air  Force  officials  state  this 
demonstration  greatly  reduced  technology  risks  and 
informed  the  preliminary  design  review  as  well  as 
the  November  2016  decision  to  enter  into  system 
development.  The  program's  acquisition  strategy 
enables  the  Air  Force  to  continue  testing  EPAWSS 
early  in  system  development  to  inform  the  critical 
design  review  and  to  ensure  system  maturity  in  time 
to  meet  the  required  entry  criteria  for  the  low-rate 
production  decision,  which  includes  completing  an 
independent  operational  assessment  of  EPAWSS 
performance. 
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Lead  Component:  Air  Force 


Common  Name:  F-22  Inc  3.2B  Mod 


F-22  Increment  3.2B  Modernization  (F-22  Inc  3.2B  Mod) 

The  Air  Force's  F-22  Raptor  is  a  stealthy  air-to-air 
and  air-to-ground  fighter/attack  aircraft.  The  Air 
Force  established  an  F-22  modernization  and 
improvement  program  in  2003  to  add  enhanced  air- 
to-ground,  information  warfare,  reconnaissance, 
and  other  capabilities,  and  to  improve  the  reliability 
and  maintainability  of  the  aircraft.  Increment  3.2B, 
the  fourth  increment  of  the  modernization  program, 
was  initially  managed  as  part  of  the  F-22  baseline 
program,  but  is  now  managed  as  a  separate  major 
defense  acquisition  program. 


Source:  U.S.  Air  Force. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017  dollars  in  millions) 

Prime  contractor:  Lockheed  Martin 

As  of 

Latest 

Percent 

Program  office:  Wright  Patterson  AFB, 

06/2013 

08/2016 

change 

OH 

Research  and  development  cost 

$1,268.0 

$1,179.4 

-7.0% 

Funding  needed  to  complete: 

Procurement  cost 

$357.5 

$365.0 

2.1% 

R&D:  $69.5  million 

Total  program  cost 

$1,625.5 

$1,544.5 

-5.0% 

Procurement:  $271.0  million 

Program  unit  cost 

$10,694 

$10,161 

-5.0% 

Total  funding:  $340.4  million 

Total  quantities 

152 

152 

0.0% 

Procurement  quantity:  108 

Acquisition  cycle  time  (months) 

72 

75 

4.2% 

Increment  3.2B  entered  production  in  August 
201 6  with  its  one  critical  technology,  a  geolocation 
algorithm,  mature  and  its  design  stable.  Prior  to 
production  start,  the  program  office  met  DOD 
criteria  for  manufacturing  readiness  but  did  not 
demonstrate  manufacturing  processes  to  be  in 
statistical  control,  which  is  inconsistent  with  best 
practices.  According  to  program  officials,  flight 
testing  will  conclude  in  June  2017,  6  months  later 
than  expected  due  to  reduced  test  range 
availability,  which  may  result  in  some  cost  growth. 
The  program  is  using  an  iterative  software 
development  process  with  13  releases  of 
capability,  which  is  3  more  than  initially  planned. 


Page  141 


Attainment  of  Product  Knowledge 

As  of  January  2017 

STATUS  CURRENT 
AT  STATUS 

Resources  and  requirements  match 

DEVELOPMENT 

•  Demonstrate  all  critical  technologies  are  very  close  to 

START 

• 

• 

final  form,  fit,  and  function  within  a  relevant  environment 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 

o 

function  within  a  realistic  environment 

• 

•  Complete  a  system-level  preliminary  design  review 

• 

• 

_ 

Product  design  is  stable 

DESIGN  REVIEW 

•  Release  at  least  90  percent  of  design  drawings 

• 

• 

•  Test  a  system-level  integrated  prototype 

o 

• 

Manufacturing  processes  are  mature 

PRODUCTION 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

START 

o 

9  or  critical  processes  are  in  statistical  control 

O 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

O 

o 

•  Test  a  production-representative  prototype 

O 

• 

^  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

O  Knowledge  not  attained  Not  applicable 

GAO-1 7-333SP  Assessments  of  Selected  Weapon  Programs 


Common  Name:  F-22  Inc  3.2B  Mod 


F-22  Inc  3.2B  Mod  Program 

Technology  and  Design  Maturity 

The  program's  sole  identified  critical  technology,  a 
geolocation  algorithm,  is  mature  as  it  has  been  flight 
qualified  in  a  realistic  environment.  According  to 
program  officials,  the  3.2B  program  will  complete 
flight  testing  in  June  2017.  The  program 
demonstrated  design  stability  prior  to  the  October 
2015  critical  design  review  (CDR)  by  releasing  98 
percent  of  its  drawings.  Currently,  the  program  has 
released  100  percent  of  its  system  level  drawings. 
The  CDR  was  a  culmination  of  multiple  incremental 
CDRs,  with  the  October  2015  review  focused  on 
software  as  the  program  had  already  completed  its 
hardware  reviews. 

According  to  program  officials,  the  3.2B  program 
intends  to  continue  flight  testing  an  integrated, 
system  level  prototype  through  June  2017,  but  has 
encountered  challenges  accessing  adequate  test 
facilities.  The  program  is  using  an  iterative  software 
development  process  where  10  software  releases 
were  planned.  However,  due  to  interface  issues,  the 
program  added  3  additional  software  releases,  for  a 
total  of  13.  Program  officials  stated  these  releases 
were  only  for  lab  use,  but  may  add  some  cost  and 
additional  time  to  the  program’s  schedule. 

Production  Maturity 

The  Principal  Deputy  Assistant  Secretary  of  the  Air 
Force  (Acquisition  and  Logistics)  approved 
Increment  3.2B  for  production  in  August  2016,  2 
months  later  than  planned,  after  implementing  a 
production  readiness  plan  and  completing 
qualification  testing  for  the  program's  hardware 
components.  The  program  office  conducted  an 
assessment  of  the  production  readiness  in  April 
2016,  finding  95  percent  of  critical  manufacturing 
processes  met  the  DOD  standard  for  entering 
production.  However,  the  contractor  did  not 
demonstrate  manufacturing  processes  to  be  in 
statistical  control  prior  to  production  start,  which  is 
inconsistent  with  best  practices. 

Other  Program  Issues 

Program  officials  stated  flight  test  delays  are  driven 
by  reduced  test  facility  availability,  and  noted  delays 
in  the  F-35  test  program  have  impacted  test  range 
availability  for  F-22  Inc.  3.2B.  Program  officials 
anticipate  these  delays  will  result  in  some  cost  and 
schedule  growth.  Program  officials  stated  all  F-22 


Increment  3.2B  modifications  will  be  completed  by 
contractor  field  teams  at  operating  bases,  which 
they  believe  is  a  more  cost  effective  approach  than 
depot-level  maintenance.  Depot-level  maintenance 
is  typically  for  major  maintenance  and  repairs,  such 
as  overhauling,  upgrading,  or  rebuilding  parts, 
assemblies,  or  subassemblies,  and  is  usually 
performed  at  a  facility  known  as  a  depot. 

Program  Office  Comments 

The  program  office  was  provided  with  a  draft  of  this 
assessment  and  did  not  have  any  comments. 
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Lead  Component:  Air  Force 


Common  Name:  FAB-T  CPT 


Family  of  Advanced  Beyond  Line-of-Sight  Terminals  Command  Post  Terminals  (FAB-T  CPT) 

The  Air  Force's  FAB-T  program  plans  to  provide  a 
family  of  satellite  communication  terminals  for 
airborne  and  ground-based  users  to  replace  many 
legacy  communication  terminals.  The  FAB-T 
command  post  terminals  (CPT)  subprogram  is 
expected  to  provide  voice  and  data  communications 
over  military  satellite  networks  for  nuclear  and 
conventional  forces  through  ground  command  posts 
and  E-6  and  E-4  aircraft.  Another  subprogram  is 
expected  to  provide  force  element  capabilities  on  B- 
2,  B-52,  and  RC-135  aircraft.  We  assessed  the  CPT 
subprogram. 

Source:  U.S.  Air  Force. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  in  millions) 

Prime  contractor:  Raytheon 

As  of 

Latest 

Percent 

Program  office:  Bedford,  MA 

12/2007 

09/2016 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$716.1 

$1,175.3 

64.1% 

R&D:  $14.4  million 

Procurement  cost 

$962.4 

$637.6 

-33.8% 

Procurement:  $350.3  million 

Total  program  cost 

$1,678.5 

$1,812.9 

8.0% 

Total  funding:  $364.7  million 

Program  unit  cost 

$17,669 

$16,632 

-5.9% 

Procurement  quantity:  45 

Total  quantities 

95 

109 

14.7% 

Acquisition  cycle  time  (months) 

129 

207 

60.5% 

While  the  technologies  for  22  initial  FAB-T  units 
appear  mature,  testing  remains  to  be  done  to 
mature  technologies  for  final  configurations  using 
new  antennas  and  it  is  unclear  whether  FAB-T's 
design  will  remain  stable.  The  Air  Force  expects 
to  receive  its  first  deliveries  in  2017,  but  these 
initial  units  will  eventually  require  retrofitting  with 
new  antennas.  According  to  officials,  issues 
discovered  during  testing  and  their  resolution 
resulted  in  delays  in  maturing  and  stabilizing  the 
new  antenna  configurations.  The  program 
expects  to  begin  production  of  one  of  three  new 
configurations  in  spring  2017,  while  the  remaining 
two  configurations  will  not  be  ready  until  the  end 
of  2017.  The  program  is  conducting  a  cost  and 
capability  analysis  for  the  force  element  terminals 
subprogram. 
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Attainment  of  Product  Knowledge 

As  of  January  2017 

STATUS  CURRENT 
AT  STATUS 

I  Resources  and  requirements  match 

DEVELOPMENT 

•  Demonstrate  all  critical  technologies  are  very  close  to 

START 

o 

final  form,  fit,  and  function  within  a  relevant  environment 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 

o 

function  within  a  realistic  environment 

.... 

•  Complete  a  system-level  preliminary  design  review 

.... 

• 

1  Product  design  is  stable 

DESIGN  REVIEW 

•  Release  at  least  90  percent  of  design  drawings 

O 

• 

•  Test  a  system-level  integrated  prototype 

O 

• 

1  Manufacturing  processes  are  mature 

PRODUCTION 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

START 

o 

9  or  critical  processes  are  in  statistical  control 

O 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

• 

• 

•  Test  a  production-representative  prototype 

o 

o 

0  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

O  Knowledge  not  attained  Not  applicable 

GAO-1 7-333SP  Assessments  of  Selected  Weapon  Programs 


Common  Name:  FAB-T  CPT 


FAB-T  CPT  Program 

Technology  and  Design  Maturity 

While  FAB-T  entered  system  development  in  2002, 
the  Air  Force  selected  a  new  contractor  to  begin 
development  in  2012.  According  to  the  program, 
technologies  used  for  the  first  two  lots  of  low-rate 
production  units  are  now  mature,  but  it  is  unclear 
whether  the  final  design  is  stable.  The  initial  units 
use  fully  functional  modification  kits  for  existing 
antennas  but  will  eventually  be  retrofitted  as 
technologies  for  new  antennas  are  not  yet  mature. 
The  program  expected  the  new  antennas  to  be  fully 
mature  prior  to  the  second  lot  procurement  decision, 
but  delays  continue.  Officials  attributed  these  delays 
to  developmental  testing  issues.  While  testing  the 
new  antenna  designs,  the  program  reported  a 
significant  number  of  drawing  increases,  which 
indicates  design  instability.  Going  forward,  program 
officials  told  us  further  design  changes  are  possible 
for  the  new  antennas  while  they  are  completing 
developmental  testing. 

Production  Maturity 

FAB-T  received  verbal  approval  to  begin  production 
in  September  2015,  followed  by  a  formal  acquisition 
decision  in  October  2015.  At  the  time,  FAB-T  had 
not  met  best  practice  standards  for  beginning 
production,  but,  according  to  the  program,  it  met 
DOD's  standards.  The  program  ordered  10 
modification  kits  in  September  2015  and  12 
modification  kits  in  July  2016.  As  of  September 
201 6,  the  contractor  had  assembled  eight  terminals; 
however,  issues  identified  in  first  article  testing  have 
delayed  deliveries  until  2017.  According  to  officials, 
they  expect  one  of  three  new  antenna 
configurations — for  fixed  ground-based  platforms — 
to  be  ready  for  production  in  spring  2017,  at  which 
time  they  may  also  purchase  additional  modification 
kits  for  existing  airborne  antenna.  Officials  do  not 
expect  the  final  two  configurations — new  antennas 
for  airborne  and  transportable  ground-based 
platforms — to  be  ready  until  the  end  of  2017. 

FAB-T  is  authorized  to  purchase  more  than  60 
percent  of  its  total  units  during  low-rate  production. 
Generally,  programs  must  provide  a  rationale  if  low- 
rate  production  quantities  will  exceed  10  percent  of 
total  quantities.  Officials  said  these  units  are 
required  to  demonstrate  initial  operational  capability 
by  the  end  of  2019. 


Other  Program  Issues 

In  July  2015,  DOD  separated  the  FAB-T  program 
into  two  subprograms:  CPT  and  force  element 
terminals  (FET).  Currently,  only  the  CPT 
subprogram  is  in  development  and  production.  In 
July  2016,  the  Air  Force  submitted  a  strategy  for 
achieving  the  FET  requirements  to  the  Secretary  of 
Defense.  In  September  2016,  the  program  began  a 
cost  and  capability  analysis,  which  it  expects  to 
complete  by  September  2017.  Until  the  FET 
subprogram  is  executed,  FAB-T  cannot  achieve  its 
planned  capabilities  that  are  based  on  the 
interaction  of  bomber  aircraft  with  intelligence, 
surveillance,  and  reconnaissance  aircraft  and  CPTs. 

FAB-T  is  designed  to  communicate  through  the 
Advanced  Extremely  High  Frequency  (AEHF) 
network  of  satellites.  Since  FAB-T  is  not  yet  fielded 
while  three  of  these  satellites  have  already  been 
launched,  the  lack  of  synchronization  between  the 
two  programs  has  resulted  in  the  underutilization  of 
these  costly  satellite  capabilities.  By  the  time  FAB-T 
achieves  initial  operational  capability  in  2019,  one  of 
the  AEHF  satellites  will  have  been  operating  for 
nine  years,  over  half  of  its  projected  14-year 
operational  lifetime. 

Program  Office  Comments 

FAB-T  officials  stated  the  contractor  continues 
development  of  the  new  ground  fixed,  ground 
transportable,  and  airborne  antennas,  and 
augments  its  workforce  as  needed  by  adding 
subject  matter  experts  to  address  technical  issues. 
Although  the  program  has  experienced  delays  in 
development  of  the  new  antennas,  it  has  achieved 
success  in  government-led  risk  reduction  testing.  A 
FAB-T  terminal  was  operated  by  space  operations 
personnel  in  successful  execution  of  over-the-air 
satellite  telemetry,  tracking,  and  control  testing.  All 
test  objectives  were  successfully  completed,  which 
included  all  required  active  command  functions  for 
control  of  Milstar  and  AEHF  satellites.  In  addition, 
the  program  office  performed  a  series  of  free-field 
high-altitude  electromagnetic  pulse  events 
simulating  68  nuclear  air  bursts,  demonstrating  the 
FAB-T  will  operate  through  electromagnetic  effects. 
The  program  continues  to  actively  visit  sites  in 
preparation  for  fielding  FAB-T  terminals  in  2017. 
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Lead  Component:  Air  Force 


Common  Name:  GPS  III 


Global  Positioning  System  III  (GPS  III) 

The  Air  Force's  Global  Positioning  System  (GPS)  III 
program  plans  to  develop  and  field  a  new 
generation  of  satellites  to  supplement  and 
eventually  replace  the  GPS  satellites  currently  in 
use.  Other  programs  are  developing  the  related 
ground  system  and  user  equipment.  GPS  III  is 
intended  to  provide  capabilities  for  a  stronger 
military  navigation  signal  to  improve  jamming 
resistance  and  a  new  civilian  signal  that  will  be 
interoperable  with  foreign  satellite  navigation 
systems. 


Source:  U.S.  Air  Force. 
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Program  Essentials 


Program  Performance  (fiscal  year  2017  dollars  in  millions) 


Prime  contractor:  Lockheed  Martin 
Program  office:  El  Segundo,  CA 
Funding  needed  to  complete: 

R&D:  $183.7  million 
Procurement:  $476.3  million 
Total  funding:  $660.1  million 
Procurement  quantity:  0 


As  of 

Latest 

Percent 

05/2008 

09/2016 

change 

Research  and  development  cost 

$2,738.0 

$3,225.6 

17.8% 

Procurement  cost 

$1,537.2 

$2,512.6 

63.5% 

Total  program  cost 

$4,275.2 

$5,738.3 

34.2% 

Program  unit  cost 

$534,401 

$573,827 

7.4% 

Total  quantities 

8 

10 

25.0% 

Acquisition  cycle  time  (months) 

NA 

NA 

NA 

We  could  not  calculate  acquisition  cycle  times  for  the  first  increment  of  the  GPS  III  program  because 
initial  operational  capability  will  not  occur  until  satellites  from  a  future  increment  are  fielded. 


The  GPS  III  program  currently  reports  mature 
technologies  and  a  stable  design,  but 
manufacturing  processes  are  not  yet  in  statistical 
control.  After  a  January  2016  rebaseline  of  the 
program's  cost  and  schedule,  the  program 
identified  deficiencies  with  subsystem 
components,  which  have  contributed  to  new 
delivery  delays  of  satellite  equipment.  The 
program  has  taken  steps  to  mitigate  these  and 
other  deficiencies,  but  now  projects  the  first 
satellite's  “available  for  launch”  date  will  be 
delayed  from  August  2016  to  February  2017  and 
additional  delays  are  likely  for  the  next  seven 
satellites.  Further,  following  extensive  delays  to 
the  GPS  III  ground  system — the  Next  Generation 
Operational  Control  System  (OCX) — the  Air  Force 
now  plans  to  accept  delivery  of  the  first  eight 
satellites  before  operational  testing  confirms  the 
satellite’s  modernized  signal  capabilities. 
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Attainment  of  Product  Knowledge 


As  of  January  2017 


Resources  and  requirements  match 


STATUS  CURRENT 
AT  STATUS 


Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

Complete  a  system-level  preliminary  design  review 


DEVELOPMENT 

START 


o 


Product  design  is  stable 


Release  at  least  90  percent  of  design  drawings 
Test  a  system-level  integrated  prototype 


DESIGN  REVIEW 


o 


Manufacturing  processes  are  mature 


Demonstrate  Manufacturing  Readiness  Level  of  at  least 
9  or  critical  processes  are  in  statistical  control 


PRODUCTION 

START 


o  o 


Demonstrate  critical  processes  on  a  pilot  production  line  o  • 

Test  a  production-representative  prototype  o  o 


9  Knowledge  attained 
O  Knowledge  not  attained 


Information  not  available 
Not  applicable 


Common  Name:  GPS  III 


GPS  III  Program 

Technology,  Design,  and  Production  Maturity 

The  GPS  III  program  currently  reports  all  eight  of  its 
critical  technologies  are  mature  and  the  design  is 
stable.  In  2016,  the  program  encountered  technical 
issues  involving  capacitors — devices  used  to  store 
energy  and  release  it  as  electrical  power.  In 
investigating  the  capacitor  failures,  the  program 
discovered  the  contractor  skipped  a  key 
developmental  step  because  a  subcontractor  had 
not  conducted  qualification  testing  for  the 
capacitor's  operational  use  in  GPS  III  satellites.  In 
response,  the  program  conducted  production  lot 
reliability  testing  and  design  qualification  testing  of 
the  capacitors.  Design  qualification  successfully 
concluded  in  December  2016.  However,  test  set-up 
problems  invalidated  the  reliability  testing  of  the 
production  lot  from  which  the  failed  capacitors 
originated.  The  program  decided  to  assume  risk  and 
proceed  “as  is”  with  satellite  1 ,  despite  it  being  fitted 
mostly  with  capacitors  from  that  lot.  For  satellites  2 
and  3,  the  program  ordered  replacement  of 
capacitors  from  the  lot.  The  capacitor  issue  has  also 
contributed  to  significant  delays  in  the 
subcontractor’s  delivery  of  navigation  payload 
components,  such  as  the  mission  data  unit  and 
transmitters.  The  program  now  projects  a  more  than 
6  month  delay  to  the  first  satellite's  “available  for 
launch”  date. 

To  prove  out  production  processes  prior  to 
integrating  and  testing  the  first  space  vehicle,  the 
program  tested  a  system-level  integrated  prototype 
that  includes  all  key  subsystems  and  components 
but  contains  less  built-in  redundancy  than  the  final 
configuration.  Although  the  program  previously 
reported  a  manufacturing  process  maturity  level  that 
indicated  processes  were  in  statistical  control,  the 
program  has  recently  reported  a  lower  level  of 
maturity,  indicating  production  processes  are  not  yet 
fully  in  statistical  control.  While  this  reported 
maturity  level  is  high  enough  to  meet  DOD's 
standards  for  production  start,  it  does  not  meet  the 
standard  recommended  by  best  practices. 

Other  Program  Issues 

Due  to  previously  reported  satellite  launch 
availability  delays,  the  program  rebaselined  its  cost 
and  schedule  in  January  2016.  This  rebaselining 
occurred  prior  to  the  February  2016  capacitor 
deficiencies,  and,  because  of  those  and  other 


subsequent  technical  issues,  the  contractor  has 
now  used  up  all  available  extra  time  in  the 
program's  schedule.  The  contractor  is  now 
projecting  deliveries  of  satellites  1  through  8  will  be 
delayed,  on  average,  by  about  6  months.  The 
Defense  Contract  Management  Agency  is 
projecting  an  additional  delivery  delay  of  9  months, 
on  average,  for  each  satellite  beyond  what  the 
contractor  now  forecasts.  According  to  contractor 
representatives,  they  are  working  with  the 
navigation  payload  contractor  to  reduce  the  length 
of  these  delays. 

Technical  issues  with  both  the  GPS  III  satellite  and 
the  OCX  Block  0  launch  control  and  checkout 
system  have  combined  to  place  the  planned  March 
201 8  launch  date  for  the  first  GPS  III  satellite  at  risk. 
In  order  to  launch  in  March  2018,  the  GPS  III  and 
OCX  programs  restructured  their  joint  pre-launch 
integrated  testing,  reducing  the  campaign  from  52 
weeks  to  42  weeks.  Since  then,  a  new  4  month 
delay  to  OCX  readiness  to  begin  launch  preparation 
has  forced  additional  campaign  restructuring. 

Because  of  extensive  delays  to  OCX,  the  GPS  III 
program  is  projecting  to  have  delivered  at  least  the 
first  eight  satellites  and  to  have  awarded  a  new 
contract  for  additional  GPS  III  satellites  before 
operational  testing  of  the  GPS  III  satellite  with  OCX 
Block  1  confirms  the  satellite's  modernized  signal 
capabilities. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  noted  the  Air  Force  placed  the  first 
GPS  III  satellite  into  storage  on  February  27,  2017, 
after  successfully  completing  all  planned  test  and 
integration  activities.  The  Air  Force  deferred 
declaring  the  first  GPS  III  satellite  available  for 
launch  due  to  a  review  of  the  Lockheed  Martin 
A2100  propulsion  sub-system.  This  review  involved 
multiple  space  programs.  The  GPS  program  office 
is  participating  in  the  review  and  continues  to 
monitor  for  any  impacts  to  the  planned  GPS  III  initial 
launch  capability  in  Spring  2018.  The  program  office 
also  provided  technical  comments,  which  were 
incorporated  where  deemed  appropriate. 


Page  146 


GAO-1 7-333SP  Assessments  of  Selected  Weapon  Programs 


Lead  Component:  Air  Force 


Common  Name:  KC-46A 


KC-46  Tanker  Modernization  Program  (KC-46A) 


The  Air  Force's  KC-46  program  plans  to  convert  an 
aircraft  designed  for  commercial  use  into  an  aerial 
refueling  tanker  for  operations  with  Air  Force,  Navy, 
Marine  Corps,  and  allied  aircraft.  The  program  is  the 
first  of  three  planned  phases  to  replace  the  Air 
Force's  aging  aerial  refueling  tanker  fleet.  The  KC- 
46  has  been  designed  to  improve  on  the  KC-135's 
refueling  capacity,  efficiency,  capabilities  for  cargo 
and  aeromedical  evacuation,  and  to  integrate 
defensive  systems. 


Source:  ©  2016  Boeing  Company  -  Photo  by  Paul  Weatherman. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  In  millions) 

Prime  contractor:  Boeing 

As  of 

Latest 

Percent 

Program  office:  Wright  Patterson  AFB, 

02/2011 

06/2016 

change 

OH 

Research  and  development  cost 

$7,432.2 

$6,531.3 

-12.1% 

Funding  needed  to  complete: 

Procurement  cost 

$36,089,9 

$34,266.5 

-5.1% 

R&D:  $283.0  million 

Total  program  cost 

$47,534.9 

$43,626.5 

-8.2% 

Procurement:  $30,455.3  million 

Program  unit  cost 

$265,558 

$243,724 

-8.2% 

Total  funding:  $33,098.9  million 

Total  quantities 

179 

179 

0.0% 

Procurement  quantity:  156 

Acquisition  cycle  time  (months) 

78 

92 

1 7.9% 

In  August  2016,  the  KC-46  program  entered  low- 
rate  production  with  its  three  critical  technologies 
fully  mature.  Although  the  Air  Force  considered 
the  KC-46  design  stable  at  the  July  2013  critical 
design  review,  Boeing  later  discovered  wiring 
issues  that  resulted  in  several  changes  to  the 

Attainment  of  Product  Knowledge 

As  of  January  2017 

STATUS  CURRENT 
AT  STATUS 

Resources  and  requirements  match 

•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

DEVELOPMENT 

START 

• 

• 

aircraft’s  wiring  system.  The  wiring  design  was 
finalized  in  March  2016  after  Boeing  had  started 
producing  aircraft.  Boeing  plans  to  correct  the 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

o 

• 

•  Complete  a  system-level  preliminary  design  review 

o 

• 

wiring  on  already  produced  aircraft  prior  to 

Product  design  is  stable 

delivery  to  the  Air  Force.  Under  the  terms  of  the 

•  Release  at  least  90  percent  of  design  drawings 

DESIGN  REVIEW 

development  contract,  Boeing  bears  the  cost  of 
correcting  these  deficiencies.  According  to  Air 

Force  and  Boeing  officials,  Boeing  will  not  make 
contractually-required  deliveries  at  the  required 
assets  availability  date  in  August  2017.  Instead, 

# 

• 

•  Test  a  system-level  integrated  prototype 

O 

• 

Manufacturing  processes  are  mature 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

9  or  critical  processes  are  in  statistical  control 

PRODUCTION 

START 

O 

o 

Boeing  plans  to  deliver  the  first  18  aircraft  by 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

• 

• 

February  201 8,  with  the  wing  aerial  refueling  pods 

•  Test  a  production-representative  prototype 

• 

• 

following  in  October  2018. 

^  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

O  Knowledge  not  attained  Not  applicable 
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Common  Name:  KC-46A 


KC-46A  Program 

Technology  and  Design  Maturity 

The  KC-46's  three  critical  technologies — two 
software  modules  related  to  situational  awareness 
and  a  three-dimensional  display  that  allows  the 
crew  to  monitor  aerial  refueling  activities — are  fully 
mature.  At  its  July  2013  critical  design  review 
(CDR),  the  program  had  released  over  90  percent 
of  its  design  drawings.  However,  following  CDR, 
Boeing  discovered  aircraft  wiring  deficiencies  that 
have  required  it  to  re-design  the  wiring  system  to 
resolve  separation  issues.  After  several 
modifications,  Boeing  finalized  the  wiring  design  in 
March  2016. 

As  of  December  201 6,  Boeing  had  completed  about 
half  of  the  planned  KC-46  developmental  testing. 
Until  this  testing  completes,  Boeing  may  find 
additional  technical  issues  that  may  require  design 
changes.  For  example,  during  demonstration  flights, 
Boeing  found  a  technical  issue  with  the  refueling 
boom  that  delayed  the  production  decision  by  4 
months. 

Production  Maturity 

Boeing  has  manufactured  four  development  aircraft 
and  has  begun  producing  the  first  12  low-rate  initial 
production  aircraft.  Boeing  is  correcting  the  wiring 
on  already  produced  low-rate  initial  production 
aircraft  by  incorporating  the  final  wiring  design. 
Under  the  terms  of  the  development  contract, 
Boeing  bears  the  cost  of  correcting  these 
deficiencies.  Program  officials  state  they  will 
continue  to  monitor  and  assess  production  maturity 
leading  up  to  the  full-rate  production  decision  in 
2019.  The  program  intends  to  purchase  more  than 
25  percent  of  its  total  aircraft  during  low-rate  initial 
production.  Generally,  programs  must  provide  a 
rationale  if  low-rate  initial  production  quantities  are 
going  to  exceed  1 0  percent  of  total  quantities.  The 
Under  Secretary  of  Defense  for  Acquisition, 
Technology  and  Logistics  approved  this  acquisition 
strategy  for  the  KC-46  program  to  avoid  a  break  in 
the  production  line. 

Other  Program  Issues 

The  Under  Secretary  of  Defense  for  Acquisition, 
Technology  and  Logistics  approved  the  KC-46 
program  to  enter  low-rate  initial  production  in 
August  201 6,  one  year  later  than  originally  planned. 
Boeing  has  proposed  modifying  its  contract  to  delay 


delivery  dates  to  account  for  testing  and  other 
delays  it  is  currently  experiencing,  including 
qualifying  the  wing  aerial  refueling  pod  drogue 
systems,  which  are  used  to  refuel  two  Navy  or  allied 
aircraft  simultaneously.  According  to  program 
officials,  Boeing  plans  to  deliver  the  first  18  aircraft 
with  refueling  booms  and  centerline  drogue  systems 
by  February  2018.  Wing  aerial  refueling  pod 
components  will  be  delivered  in  October  2018.  At 
that  point,  Boeing  will  have  delivered  these  items  14 
months  after  their  required  availability  in  August 
2017.  Program  officials  are  currently  negotiating  the 
necessary  contract  modifications  related  to  this 
delay. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  technical  comments,  which 
were  incorporated  where  deemed  appropriate. 
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Common  Name:  MGUE  Increment  1 


Military  GPS  User  Equipment  (MGUE)  Increment  1 

The  Air  Force's  MGUE  program  plans  to  develop 
GPS  receivers  compatible  with  the  military's  next- 
generation  GPS  signal,  Military-Code.  The 
modernized  receivers  will  provide  U.S.  forces  with 
enhanced  position,  navigation,  and  time 
capabilities,  while  improving  resistance  to  existing 
and  emerging  threats,  such  as  jamming.  The 
program  consists  of  two  increments.  Increment  1, 
assessed  here,  leverages  technologies  from  the 
Modernized  User  Equipment  program  to  develop 
receivers  for  aviation,  maritime,  and  ground 
platforms. 


Concept  System  development 
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▲ 

Program 

Preliminary 

Development 

GAO 

start 

design  review 

start 

review 
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Program  Essentials 

Program  Performance  (fiscal  year  2017  dollars  in  millions) 

Prime  contractors:  L-3,  Raytheon, 

As  of 

Latest 

Percent 

Rockwell  Collins  International 

10/2016 

change 

Program  office:  El  Segundo,  CA 

Research  and  development  cost 

NA 

$1,143.5 

NA 

Funding  needed  to  complete: 

Procurement  cost 

NA 

NA 

NA 

R&D:  $585.1  million 

Total  program  cost 

NA 

$1,143.5 

NA 

Procurement:  $0.0  million 

Program  unit  cost 

NA 

NA 

NA 

Total  funding:  $585.1  million 

Total  quantities 

NA 

0 

NA 

Procurement  quantity:  0 

Acquisition  cycle  time  (months) 

NA 

NA 

NA 

MGUE  entered  system  development  in  January 
201 7  with  four  of  five  critical  technologies  mature, 
but  independent  reviews  raised  concerns  about 
past  technology  maturity  assessments.  MGUE 
recently  began  building  finished  hardware  for 
testing  and  integration,  but  contractors  will 
continue  to  incrementally  develop  and  deliver 
software.  Integration  testing — during  which  time 
problems  may  be  discovered — will  not  be 
completed  until  2021 ,  meaning  the  services  are 
unlikely  to  have  sufficient  knowledge  on  which  to 
base  a  procurement  decision.  The  National 
Defense  Authorization  Act  for  Fiscal  Year  2011 
generally  prohibits  DOD  from  obligating  or 
expending  funds  for  GPS  user  equipment  after 
fiscal  year  2017  unless  that  equipment  is  capable 
of  receiving  military-code.  The  Secretary  of 
Defense  may  waive  this  limitation  under  certain 
circumstances,  or  certain  exceptions  may  apply. 


PRODUCTION 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 
9  or  critical  processes  are  in  statistical  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 

^  Knowledge  attained  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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MGUE  Increment  1  Program 

Technology  and  Design  Maturity 

The  MGUE  Increment  1  program  entered  system 
development  in  January  2017  and  has  assessed 
four  of  its  five  critical  technologies — military-code 
acquisition  engine,  military-code  cryptography, 
selective  availability  anti-spoofing  module 
functionality,  and  anti-tamper — as  mature.  Anti¬ 
spoof  is  nearing  maturity.  However,  independent 
reviews  questioned  past  assessments  of  the 
program's  technology  maturity.  MGUE  maturity 
levels  were  last  formally  assessed  during  a 
November  2014  independent  technology  readiness 
assessment.  At  that  time,  DOD's  Director, 
Operational  Test  and  Evaluation  expressed 
concerns  about  one  of  the  Air  Force's  test 
demonstrations  and  emphasized  in  a  memorandum 
to  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology  and  Logistics  the  Air  Force  had 
overstated  MGUE  maturity,  and  the  demonstration 
results  were  more  mixed  than  the  service  indicated. 
In  September  2015,  the  Air  Force  Operational  Test 
and  Evaluation  Center  initiated  an  operational 
assessment  of  MGUE,  but  then  terminated  the 
assessment  before  completing  it  due  to  immature 
test  articles  and  the  lack  of  developmental 
performance  testing  and  data.  A  senior  program 
official  indicated  the  software  immaturity  in  MGUE 
test  articles  was  largely  responsible  for  the  poor 
performance  during  the  operational  assessment. 

In  2016,  the  program  began  providing  final  test 
articles  to  ground,  air,  and  maritime  platform  offices 
to  support  risk  reduction  and  subsystem  integration 
activities.  These  test  articles  are  finished  hardware 
components,  but  the  software  they  contain  is 
incomplete.  The  three  MGUE  contractors  will 
develop  and  deliver  additional  software  increments, 
which  will  be  uploaded  to  the  existing  hardware  to 
support  further  test  activities,  including  integration 
testing  with  the  services'  lead  platforms.  MGUE  is 
scheduled  to  complete  this  phase  of  testing,  which 
commonly  reveals  problems,  in  2021 . 

Other  Program  Issues 

MGUE  will  not  have  a  production  decision  and  the 
program's  acquisition  strategy  does  not  provide  for 
equipment  procurement  beyond  the  final  test 
articles.  Once  operational  testing  is  complete,  the 
military  services  will  assume  responsibility  for 


developing  MGUE  technology  to  a  production-ready 
status,  as  well  as  funding,  procuring,  and  integrating 
the  components  on  their  platforms. 

The  National  Defense  Authorization  Act  for  Fiscal 
Year  2011  generally  prohibits  DOD  from  obligating 
or  expending  funds  for  GPS  user  equipment  after 
fiscal  year  201 7  unless  that  equipment  is  capable  of 
receiving  military-code  signals.  However,  due  to 
delays  in  the  program's  operational  testing 
schedule,  the  military  services  are  unlikely  to  have 
sufficient  knowledge  to  begin  MGUE  procurement  at 
the  start  of  fiscal  year  2018.  The  Secretary  of 
Defense  may  waive  this  funding  limitation  under 
certain  circumstances,  or  exceptions  may  apply. 
Officials  from  each  military  service  indicated  they 
are  likely  to  request  a  waiver. 

Program  Office  Comments 

According  to  program  officials,  MGUE  is  conducting 
a  test  program  intended  to  uncover  risk  as  early  as 
possible  and  verify  the  functional  baseline,  including 
testing  on  platforms  and  in  software  integration  labs. 
Program  officials  report  performance  will  be 
demonstrated  through  testing  and  will  support  the 
services  need  to  procure  MGUE  receivers.  Officials 
further  noted  program  managers  for  non-lead 
platforms  may  make  decisions  on  when  to 
incorporate  MGUE  Increment  1  technology  prior  to 
the  completion  of  the  program. 
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Next  Generation  Operational  Control  System  (GPS  OCX) 

The  Air  Force's  GPS  OCX  is  primarily  a  software 
development  program  to  replace  the  existing 
ground  control  system.  Intended  to  ensure  reliable, 
secure  delivery  of  position  and  timing  information  to 
military  and  civilian  users,  OCX  software  will  be 
delivered  in  blocks,  each  providing  upgrades  as 
they  become  available.  We  assessed  the  first  three 
blocks:  Block  0  for  initial,  limited  testing  of  new 
satellites;  Block  1  for  satellite  control  and  basic 
military  signals;  and  Block  2  for  modernized  military 
and  additional  navigation  signals. 


Source:  U.S.  Air  Force. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  In  millions) 

Prime  contractor:  Raytheon 

As  of 

Latest 

Percent 

Program  office:  El  Segundo,  CA 

11/2012 

09/2016 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$3,591.8 

$5,498.4 

53.1% 

R&D:  $2,183.4  million 

Procurement  cost 

$0.0 

$0.0 

0.0% 

Procurement:  $0.0  million 

Total  program  cost 

$3,591.8 

$5,498.4 

53.1% 

Total  funding:  $2,183.4  million 

Program  unit  cost 

$3,591,802 

$5,498,444 

53.1% 

Procurement  quantity:  1 

Total  quantities 

1 

1 

0.0% 

Acquisition  cycle  time  (months) 

55 

116 

110.9% 

The  GPS  OCX  program  considers  all  of  its  14 
critical  technologies  mature.  The  program 
awarded  a  development  contract  in  early  2010 
and  entered  system  development  in  late  2012. 
The  next  year,  the  Air  Force  paused  OCX 
development  to  address  problems  it  believed 
were  causing  significant  cost  and  schedule 
growth,  and  acknowledged  program  and 
contractor  understanding  of  key  requirements, 
especially  cybersecurity,  was  deficient.  The  Air 
Force  resumed  OCX  development  in  2015,  but 
the  program  continued  to  experience  cost  and 
schedule  growth.  Last  year,  the  Air  Force 
determined  OCX  would  breach  a  statutory  critical 
unit  cost  growth  threshold,  and  require 
recertification  to  continue  development.  DOD 
recertified  the  program  in  October  2016,  with  cost 
growth  of  over  50  percent  from  the  original 
program  baseline  and  a  24-month  schedule 
extension. 
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Attainment  of  Product  Knowledge 


As  of  January  2017 


Resources  and  requirements  match 


STATUS  CURRENT 
AT  STATUS 


•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

DEVELOPMENT 

START 

• 

• 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

o 

o 

•  Complete  a  system-level  preliminary  design  review 

• 

• 

Product  design  is  stable 

•  Release  at  least  90  percent  of  design  drawings 

DESIGN  REVIEW 

•  Test  a  system-level  integrated  prototype 

Manufacturing  processes  are  mature 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

PRODUCTION 

START 

9  or  critical  processes  are  in  statistical  control 
Demonstrate  critical  processes  on  a  pilot  production  line 
Test  a  production-representative  prototype 


^  Knowledge  attained 
O  Knowledge  not  attained 


Information  not  available 
Not  applicable 


Common  Name:  GPS  OCX 


GPS  OCX  Program 

Technology  and  Design  Maturity 

The  OCX  program  currently  assesses  its  14  critical 
technologies  as  mature.  The  OCX  program  does 
not  track  the  metrics  we  use  to  measure  design 
maturity,  such  as  the  number  of  releasable  design 
drawings,  as  it  is  primarily  a  software  development 
effort. 

OCX  entered  system  development  in  November 
2012  with  all  14  critical  technologies  nearing 
maturity.  This  milestone  was  preceded  by 
development  contract  award  in  February  2010  and 
preliminary  design  review  in  August  2011.  In  201 3, 
following  high  cost  and  schedule  growth,  the  Air 
Force  paused  OCX  development  to  identify  and 
address  the  root  causes  thereof.  According  to 
program  officials,  the  Air  Force  and  the  contractor 
had  poorly  understood  key  requirements, 
particularly  for  cybersecurity,  which  led  to  difficulty  in 
developing  effective  software.  Specifically,  OCX 
contractor  representatives  said  they  did  not  fully 
understand  the  cybersecurity  implementation 
requirements  designed  to  help  ensure  system 
resistance  to,  and  operation  during,  cyber-attacks. 

In  2015,  the  Air  Force  restarted  OCX  software 
development  activities  prior  to  fully  identifying  and 
addressing  root  causes  and  without  realistic  cost 
and  schedule  estimates.  DOD  recently  conducted  a 
new  root  cause  analysis,  which  identified  that  (1) 
external  factors  drove  the  program  to  an  unrealistic 
schedule,  (2)  costs  to  fully  implement  information 
assurance  requirements  were  underestimated,  and 
(3)  both  the  contractor  and  government  had 
performed  poorly.  In  December  2015,  2  months 
after  re-baselining  the  schedule,  the  Office  of  the 
Under  Secretary  of  Defense  for  Acquisitions, 
Technology  and  Logistics  directed  a  high-risk,  24- 
month  schedule  extension  to  Block  1.  OCX  is  also 
scheduled  to  repeat  the  milestone  review 
associated  with  entry  into  system  development  by 
June  2017,  after  the  contractor  develops  new  cost 
and  schedule  baselines. 

Other  Program  Issues 

The  program  remains  at  high-risk  of  cost  growth, 
schedule  delays,  and  performance  shortfalls  to  the 
deliveries  of  Block  0  and  Block  1 .  The  revised 
schedule  optimistically  assumes  higher  levels  of 
software  coding  productivity  than  the  contractor  has 


previously  accomplished,  and  reductions  in  defect 
rates  using  an  entirely  new  software  development 
methodology.  Further,  the  schedule  has  insufficient 
margin  for  unexpected  issues.  In  addition,  although 
the  contractor  has  nearly  doubled  the  software 
engineers  working  on  OCX,  it  is  having  difficulty 
training  those  staff  on  software  development. 

Delivery  of  OCX  Block  0  may  affect  the  program 
created  to  offset  OCX  delays,  known  as  the 
Contingency  Operations  (COps)  program.  The 
COps  program  is  intended  to  modify  the  existing 
GPS  ground  control  system  to  allow  it  to  control 
GPS  III  satellites  after  launch.  The  OCX  Block  0 
contractor  delivery  has  slipped  to  September  2017, 
requiring  some  replanning  with  COps  to  prevent 
disrupting  testing  plans  for  common  test  equipment. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment, 
program  officials  provided  the  following  information. 
Block  0  acceptance  is  on  track  for  September  201 7. 
The  Air  Force  was  directed  at  the  December  2015 
Deep  Dive  program  review  to  plan  to  have  the  Block 
1  software  ready  to  transition  to  operations  by  July 
2021 .  This  date  is  being  reassessed  as  part  of  the 
program’s  breach  of  a  statutory  critical  unit  cost 
growth  threshold  and  subsequent  program  re¬ 
baselining  planned  for  the  summer  201 7.  All 
remaining  Block  2  scope  has  been  rephased  to 
deliver  concurrently  with  Block  1 ,  so  there  is  no 
longer  a  separate  Block  2  delivery  for  OCX. 
Technical  comments  were  also  provided  by  the 
program  office,  which  were  incorporated  where 
deemed  appropriate. 
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Lead  Component:  Air  Force 


Common  Name:  SDB  II 


Small  Diameter  Bomb  Increment  II  (SDB  II) 


The  Air  Force's  Small  Diameter  Bomb  Increment  II 
(SDB  II)  is  designed  to  provide  attack  capability 
against  mobile  targets  in  adverse  weather  from 
extended  range.  It  combines  radar,  infrared,  and 
semiactive  laser  sensors  in  a  tri-mode  seeker  to 
acquire,  track,  and  engage  targets.  It  uses  airborne 
and  ground  data  links  to  update  target  locations,  as 
well  as  GPS  and  an  inertial  navigation  system  to 
ensure  accuracy.  SDB  II  will  be  integrated  with  F- 
15E,  F/A-18E/F,  and  F-35  aircraft,  among  others. 


Source:  ©  2009  Raytheon  Company. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  in  millions) 

Prime  contractor:  Raytheon 

As  of 

Latest 

Percent 

Program  office:  Eglin  AFB,  FL 

10/2010 

08/2016 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$1,784.1 

$1,832.2 

2.7% 

R&D:  $497.2  million 

Procurement  cost 

$3,316.2 

$2,454.6 

-26.0% 

Procurement:  $2,349.3  million 

Total  program  cost 

$5,100.3 

$4,286.8 

-15.9% 

Total  funding:  $2,846.5  million 

Program  unit  cost 

$0,297 

$0,250 

-15.9% 

Procurement  quantity:  16,606 

Total  quantities 

17,163 

17,163 

0.0% 

Acquisition  cycle  time  (months) 

72 

96 

33.3% 

SDB  II  entered  production  in  June  2015  with  all  of 
its  critical  technologies  mature;  however,  its 
design  was  unstable.  The  program  still  has 
significant  testing  to  complete,  and  any  problems 
discovered  could  result  in  further  design  changes. 
Since  January  2016,  the  program  has  conducted 
25  flight  tests  and  failed  6  of  them.  These  flight 
test  failures  occurred  during  all  three  of  SDB  ll's 
attack  modes.  The  program  successfully  retested 
two  of  the  failed  tests  and  plans  to  retest  others. 
These  failures  have  contributed  to  the  program's 
6-month  delay  in  initial  operational  capability  with 
the  F-15E.  The  program  also  successfully 
completed  the  corrosive  atmosphere  testing  that 
was  required  before  awarding  a  second 
production  contract  in  September  2016.  The 
program's  critical  processes  for  production  are  in 
statistical  control. 
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Attainment  of  Product  Knowledge 


As  of  January  2017 

STATUS  CURRENT 
AT  STATUS 

I  Resources  and  requirements  match 

•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

DEVELOPMENT 

START 

• 

• 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

o 

• 

•  Complete  a  system-level  preliminary  design  review 

• 

• 

1  Product  design  is  stable 

•  Release  at  least  90  percent  of  design  drawings 

DESIGN  REVIEW 

• 

• 

•  Test  a  system-level  integrated  prototype 

o 

• 

1  Manufacturing  processes  are  mature 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

9  or  critical  processes  are  in  statistical  control 

PRODUCTION 

START 

• 

• 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

• 

• 

•  Test  a  production-representative  prototype 

• 

• 

^  Knowledge  attained  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  SDB  II 


SDB  II  Program 

Technology  and  Design  Maturity 

SDB  ll's  four  critical  technologies — guidance  and 
control,  multi-mode  seeker,  net  ready  data  link,  and 
payload — are  mature.  These  technologies  did  not 
mature  though  until  almost  5  years  after 
development  start,  in  time  for  the  program’s  low-rate 
production  decision  in  June  2015. 

Deficiencies  in  SDB  M's  design  were  revealed  at 
production  start  as  a  result  of  qualification  and  flight 
test  failures  that  required  hardware  and  software 
changes.  Additional  changes  and  retrofits  remain 
possible  as  SDB  II  continues  developmental  testing. 
Since  January  2016,  the  program  has  conducted  25 
flight  tests  in  all  three  of  the  SDB  II  attack  modes. 
Six  of  these  tests  did  not  achieve  their  objectives. 
Specifically,  one  live  fire  test,  one  government 
confidence  test,  and  2  guided  flight  tests  failed  to 
impact  their  intended  targets;  one  laser  flight  test 
performed  as  intended,  but  the  pilot  lost  the  track  of 
the  intended  target  and  the  weapon  missed;  and 
one  coordinate  attack  flight  test  detonated  10  times 
above  the  planned  detonation  height.  The  program 
successfully  retested  the  2  guided  flight  tests  and 
plans  to  retest  both  the  laser  flight  test  and  the 
coordinate  attack  flight  test.  According  to  officials, 
the  program  has  modified  SDB  II  software  and 
hardware  to  correct  for  known  failures. 

These  testing  outcomes  contributed  to  a  6-month 
delay  in  SDB  ll’s  initial  operational  capability  with 
the  F-15E.  The  program  began  a  "government 
confidence  test"  program  in  October  2016  and  has 
conducted  6  of  the  28  shots,  all  of  which  will  occur  in 
the  normal  attack  mode.  The  program  added  these 
tests  at  the  direction  of  the  Office  of  the  Secretary  of 
Defense  to  test  against  additional,  real  world 
scenarios.  The  program  plans  to  complete  this  test 
program  within  9  months.  However,  failures  of  any 
of  the  government  confidence  tests  could  delay 
subsequent  tests. 

Production  Maturity 

The  contractor  is  scheduled  to  begin  low-rate  initial 
production  of  the  first  144  units  (Lot  1)  in  March 
2017,  following  current  production  of  SDB  II  test 
assets.  The  program  awarded  a  second  production 
contract  (Lot  2)  in  September  2016  for  250 
additional  units  and  a  third  production  contract 


(Lot  3)  in  January  2017  for  312  additional  units. 
According  to  the  program,  its  critical  manufacturing 
processes  are  in  statistical  control. 

Other  Program  Issues 

In  2016,  the  program  completed  corrosive 
atmosphere  environmental  testing  of  SDB  II,  which 
revealed  deficiencies.  This  testing  built  upon  prior 
testing  in  2014  and  2015,  which  informed  SDB  M's 
low-rate  production  decision  in  June  2015. 
According  to  officials,  this  testing  simulated  the 
bomb's  exposure  to  various  environmental 
conditions  for  an  extended  period  outside  of  its 
protective  container.  In  the  2016  testing,  SDB  II 
passed  the  required  software  tests  but  failed 
functionality  tests.  Specifically,  the  dome  cover, 
wings,  air  turbine  alternator,  and  UHF  antenna  failed 
to  deploy.  The  program  implemented  corrective 
fixes  and  then  retested  successfully.  According  to 
the  program,  this  successful  test  was  necessary 
before  the  program  could  award  the  Lot  2 
production  contract.  Officials  stated  the  design 
changes  from  the  environmental  test  will  be 
incorporated  into  Lot  2  production,  which  will  cause 
Lot  1  and  2  configurations  to  be  slightly  different. 
According  to  officials,  recent  environmental  testing 
was  mainly  to  ensure  SDB  II  survivability  after 
exposure  to  the  environment  of  Navy  ships, 
although  the  Navy  does  not  intend  to  procure  any 
bombs  until  Lot  4  production. 

Program  Office  Comments 

According  to  program  officials,  SDB  II  is 
demonstrating  effectiveness  and  lethality  in  flight 
test.  The  program  is  events-based  and  is  effectively 
resolving  technical  issues.  Recent  flight  tests 
demonstrated  successful  performance  against  two 
important  requirements:  performance  in  adverse 
weather  and  weapon  control  by  a  third  party. 
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GAO-1 7-333SP  Assessments  of  Selected  Weapon  Programs 


Lead  Component:  Air  Force 


Common  Name:  Space  Fence  Inc  1 


Space  Fence  Ground-Based  Radar  System  Increment  1 

The  Air  Force's  Space  Fence  Increment  1  program 
is  developing  a  large,  ground-based  radar  to  detect 
and  track  objects  in  low  and  medium  Earth  orbit  and 
provide  this  information  to  a  space  surveillance 
network.  Space  Fence  is  designed  to  use  high  radio 
frequencies  to  detect  and  track  more  and  smaller 
objects  than  previous  systems.  The  Air  Force 
awarded  a  development  and  production  contract  for 
the  first  site  in  June  2014,  and  included  a  second 
site  as  an  option,  which,  if  exercised,  will  be 
acquired  as  Increment  2  in  a  separate  program. 


Source:  U.S.  Air  Force. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017  dollars  in  millions) 

Prime  contractor:  Lockheed  Martin 

As  of 

Latest 

Percent 

Program  office:  Hanscom  AFB,  MA 

06/2014 

08/2016 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$1,632.1 

$1,553.3 

-4.8% 

R&D:  $219.9  million 

Procurement  cost 

$0.0 

$0.0 

0.0% 

Procurement:  $0.0  million 

Total  program  cost 

$1,632.1 

$1,553.3 

-4.8% 

Total  funding:  $219.9  million 

Program  unit  cost 

$1,632,122 

$1,553,276 

-4.8% 

Procurement  quantity:  0 

Total  quantities 

1 

1 

0.0% 

Acquisition  cycle  time  (months) 

124 

118 

-4.8% 

Since  our  last  assessment,  the  Space  Fence 
program  began  testing  an  integrated  prototype 
radar  that  includes  production-representative 
elements  and  uses  software  developed  for  the 
operational  radar.  According  to  the  Air  Force,  the 
prototype  has  demonstrated  up  to  70  percent  of 
the  operational  radar’s  capability.  The  program 
recently  delayed  its  operational  testing  and  initial 
capability  dates  due  to  challenges  with 
construction  of  the  facilities  to  house  the  radar. 
Flowever,  program  officials  report  these  delays 
will  not  affect  the  program's  ability  to  meet  its 
schedule  requirements.  To  accommodate  the 
projected  volume  of  data  generated  by  Space 
Fence,  the  Air  Force  is  developing  a  new  data 
processing  system  under  a  separate  program. 
Despite  recent  delays  in  this  program,  the  Air 
Force  expects  to  developmental^  test  the  data 
processing  system  with  Space  Fence  in  2018. 


Attainment  of  Product  Knowledge 


As  of  January  2017 

STATUS  CURRENT 
AT  STATUS 

I  Resources  and  requirements  match 

•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

DEVELOPMENT 

START 

• 

• 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

o 

• 

•  Complete  a  system-level  preliminary  design  review 

• 

• 

_ 

1  Product  design  is  stable 

•  Release  at  least  90  percent  of  design  drawings 

DESIGN  REVIEW 

• 

• 

•  Test  a  system-level  integrated  prototype 

o 

• 

1  Manufacturing  processes  are  mature 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

PRODUCTION 

START 

9  or  critical  processes  are  in  statistical  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 

^  Knowledge  attained  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  Space  Fence  Inc  1 


Space  Fence  Inc  1  Program 

Technology  Maturity 

A  February  2015  technology  readiness  assessment 
showed  all  seven  of  the  program’s  critical 
technologies  are  fully  mature.  This  maturity  was 
achieved  by  integrating  the  technologies  into  a 
prototype  radar  array — and  demonstrating  that 
array  in  an  operational  environment — in  support  of 
the  critical  design  review.  Space  Fence 
technologies  provide  capabilities  for  transmitting 
and  receiving  radar  signals  from  the  radar  array. 

Design  and  Production  Maturity 

In  early  2016,  the  program  began  testing  a 
prototype  that  uses  production-representative 
hardware  and  runs  the  software  designed  for  the 
operational  radar.  According  to  the  program,  the 
prototype  has  demonstrated  about  70  percent  of 
Space  Fence  performance  requirements.  According 
to  the  program  office,  the  prototype  is  not  a  contract 
requirement;  the  contractor  elected  to  build  it  as  part 
of  the  development  process  and  it  has  helped  them 
learn  more  about  the  installation  of  radar 
components  and  to  identify  and  correct  software 
defects. 

The  Space  Fence  program  is  tracking  the  maturity 
of  two  critical  manufacturing  processes  for 
components  of  the  radar's  transmit  and  receive 
modules.  The  contractor  has  not  demonstrated 
these  processes  to  be  in  statistical  control,  as 
recommended  by  best  practices,  but  has  achieved 
the  level  of  maturity  required  in  the  system 
development  contract.  According  to  the  program 
office,  it  is  tracking  a  manufacturing  delay  for  one 
component  of  the  radar  but  expects  to  complete 
production  and  ship  the  components  within  the 
necessary  timeline. 

Other  Program  Issues 

The  Space  Fence  program  delayed  its  operational 
testing  and  initial  operational  capability  events  due 
to  challenges  constructing  facilities  to  house  the 
radar  and  poor  performance  by  a  subcontractor. 
These  delays  eliminated  the  extra  time  built  into  the 
contractor's  schedule,  but  program  officials  stated 
the  program  remains  on  track  to  meet  its  baseline 
schedule  requirements. 

Space  Fence  Increment  1  is  expected  to  meet  the 
Air  Force's  requirements  for  initial  operational 
capability,  but  full  capability  will  only  be  achieved 
once  a  second  site  is  operational.  Development  and 


production  of  the  second  site,  which  will  represent 
an  Increment  2  program,  is  a  contract  option.  The 
program  noted  the  Air  Force  will  need  to  make  a 
preliminary  decision  on  an  Increment  2  by  summer 
2017  for  budgeting  purposes,  though  the  option  for 
a  second  site  will  not  be  exercised  until  Increment  1 
achieves  initial  operational  capability. 

The  Joint  Space  Operations  Center  (JSpOC)  at 
Vandenberg  AFB  is  acquiring  new  data  processing 
capabilities  under  its  JSpOC  Mission  System  (JMS), 
designed  in  part  to  enable  processing  of  the 
increased  volume  of  data  expected  from  Space 
Fence.  Flowever,  the  JMS  program  has  experienced 
delays  over  the  past  year.  Completion  of 
developmental  testing  of  JMS  software  has  been 
delayed  from  August  2015  to  February  2018. 
According  to  the  Space  Fence  program  office,  this 
software  is  needed  to  complete  Space  Fence 
testing  in  September  2018,  and  the  JMS  schedule 
currently  supports  this  date.  Alternatively,  if  JMS 
software  is  further  delayed,  Space  Fence 
developmental  testing  could  be  accomplished 
through  modeling  and  simulating  JMS  capabilities, 
according  to  the  Space  Fence  program.  Flowever  if 
this  option  were  used  for  operational  testing,  it  may 
cause  schedule  delays  as  it  would  require 
accreditation  of  the  model — an  activity  for  which  the 
Air  Force  has  not  budgeted  or  planned. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  noted  the  program  continues  to 
move  forward  with  minimal  impacts  to  cost  and 
schedule  since  contract  award.  It  concluded 
software  development  in  October  2016  and 
anticipated  completing  85  percent  of  radar 
manufacturing  by  the  end  of  2016.  The  program 
stated  the  construction  of  the  sensor  site  and  power 
plant  annex  facilities  on  Kwajalein  was  an  issue 
early  in  2016,  and  the  contractor  took  steps  to 
offload  work  from  an  underperforming  second  tier 
subcontractor,  including  replacing  the  construction 
subcontractor  team  lead  and  bringing  in  its  own 
senior  construction  managers.  In  December  2016, 
the  program  successfully  verified  94  percent  of  the 
298  requirements  tested  by  the  contractor.  In  2017, 
the  program  will  focus  on  completing  facility 
construction,  radar  installation  and  check  out, 
integrating  the  sensor  site  with  the  power  plant 
annex,  and  providing  power  to  the  transmit  and 
receive  arrays. 
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Lead  Component:  Air  Force 


Common  Name:  3DELRR 


Three-Dimensional  Expeditionary  Long-Range  Radar  (3DELRR) 


The  Air  Force's  3DELRR  is  being  developed  as  a 
long-range,  ground-based  sensor  for  detecting, 
identifying,  tracking,  and  reporting  aerial  targets, 
including  highly  maneuverable  and  low  observable 
targets.  The  system  intends  to  provide  real-time 
data  and  support  a  range  of  operations  in  all  types 
of  weather  and  terrain.  It  will  replace  the  Air  Force's 
AN/TPS-75  radar  system,  which  has  reached  the 
end  of  its  service  life  and  is  becoming  more  costly  to 
maintain. 


Source:  U.S.  Air  Force. 
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Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  in  millions) 

Prime  contractor:  TBD 

As  of 

Latest 

Percent 

Program  office:  Hanscom  AFB,  MA 

11/2016 

change 

Funding  needed:  (FY  201 7  to  FY  2021 ): 

Research  and  development  cost 

NA 

$483.8 

NA 

R&D:  $150.7  million 

Procurement  cost 

NA 

$938.5 

NA 

Procurement:  $267.5  million 

Total  program  cost 

NA 

$1,422.2 

NA 

Total  funding:  $418.2  million 

Program  unit  cost 

NA 

$40,636 

NA 

Procurement  quantity:  11 

Total  quantities 

NA 

35 

NA 

Acquisition  cycle  time  (months) 

NA 

163 

NA 

In  October  2014,  3DELRR  entered  system 
development  and  awarded  a  development 
contract.  At  this  point,  all  of  the  program's  critical 
technologies  were  nearing  maturity.  Flowever,  bid 
protests  delayed  the  start  of  development  work, 
and  the  Air  Force  re-opened  the  competition  in 
response  to  issues  raised  in  the  protests.  The  Air 
Force  plans  to  award  a  new  development  contract 
in  the  second  quarter  of  fiscal  year  2017,  more 
than  2  years  after  the  original  contract  award. 
Prior  to  the  start  of  development,  the  program 
took  steps  to  reduce  technical  risk  and  costs  by 
conducting  system-level  competitive  prototyping 
and  analyzing  the  tradeoffs  between  costs  and 
requirements.  The  program's  remaining  risks  may 
vary  based  on  the  design  selected  for 
development,  although  software  integration  will 
be  a  risk  regardless  of  the  contractor. 
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Attainment  of  Product  Knowledge 


As  of  January  2017 


Resources  and  requirements  match 


STATUS  CURRENT 
AT  STATUS 


•  Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 

DEVELOPMENT 

START 

• 

• 

•  Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 

o 

o 

•  Complete  a  system-level  preliminary  design  review 

• 

• 

Product  design  is  stable 

•  Release  at  least  90  percent  of  design  drawings 

DESIGN  REVIEW 

•  Test  a  system-level  integrated  prototype 

Manufacturing  processes  are  mature 

•  Demonstrate  Manufacturing  Readiness  Level  of  at  least 

PRODUCTION 

START 

9  or  critical  processes  are  in  statistical  control 
Demonstrate  critical  processes  on  a  pilot  production  line 
Test  a  production-representative  prototype 


^  Knowledge  attained 
O  Knowledge  not  attained 


Information  not  available 
Not  applicable 
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Common  Name:  3DELRR 


3DELRR  Program 

Technology  and  Design  Maturity 

The  3DELRR  program  entered  system  development 
in  October  2014  with  all  of  its  critical  technologies 
approaching  maturity.  Shortly  thereafter,  the  Air 
Force  suspended  performance  on  the  development 
contract  awarded  to  Raytheon  as  a  result  of  bid 
protests  by  Lockheed  Martin  and  Northrop 
Grumman.  In  January  2015,  GAO  dismissed  the 
protests  when  the  Air  Force  agreed  to  take 
corrective  actions  to  address  the  issues  raised.  The 
program  subsequently  re-entered  the  source 
selection  phase,  which  it  expects  to  conclude  in 
early  2017. 

Following  the  new  source  selection,  the  program 
plans  to  fully  mature  its  critical  technologies  by 
demonstrating  them  in  a  realistic  environment. 
According  to  program  officials,  the  critical 
technologies  will  vary  based  on  the  contractor 
design  selected  for  development.  Further,  as  a 
result  of  the  ongoing  source  selection  process,  the 
program  office  was  not  yet  able  to  provide  us  with 
data  on  3DELRR  design  drawings,  which  we  use  to 
evaluate  design  stability  under  our  best  practices 
criteria.  The  program  office  did,  however,  identify 
several  developmental  risks  it  is  tracking  that  could 
carry  design  implications.  For  example,  3DELRR's 
planned  design  is  software-intensive,  and  program 
officials  identified  software  development  as  a  risk 
because,  if  not  performed  adequately,  subsequent 
integration  of  hardware  and  software  could  be 
delayed.  Program  officials  also  stated  integrating 
the  extensive  amount  of  re-used  software  code 
contributed  to  the  level  of  risk,  but  noted  each 
contractor  is  planning  to  test  software  prior  to 
integration  with  the  system.  In  addition,  3DELRR  is 
expected  to  use  a  new  semiconductor  technology, 
which  relies  on  gallium  nitride-based  modules  for 
individual  radiating  elements  key  to  transmitting  and 
receiving  electromagnetic  signals,  rather  than  the 
legacy  gallium  arsenide  transmit/receive  modules. 
While  the  long-term  reliability  and  performance  of 
gallium  nitride  is  unknown  and  could  affect  radar 
sensitivity  and  power  requirements,  it  has  the 
potential  to  provide  higher  efficiency  with  lower 
power  and  cooling  demands  than  legacy 
semiconductor  technology. 


To  reduce  technical  risk  ahead  of  system 
development,  the  3DELRR  program  conducted 
system-level  competitive  prototyping,  held 
preliminary  design  reviews  with  multiple  contractors, 
and  conducted  capability  demonstrations. 
According  to  program  officials,  these  efforts  helped 
the  program  develop  critical  technologies,  refine 
technical  requirements  and  cost  estimates,  and 
assess  manufacturing  readiness. 

Other  Program  Issues 

Program  officials  stated  the  program  re-entered 
source  selection  in  May  2015  and  had  originally 
planned  to  award  a  new  system  development 
contract  in  the  second  quarter  of  fiscal  year  2016. 
Plowever,  program  officials  stated  the  contract 
award  has  been  delayed  a  year  to  the  second 
quarter  of  fiscal  year  2017,  in  part  because  the 
program  added  a  full-rate  production  option  to  the 
solicitation  in  July  2016.  The  original  development 
contract  awarded  in  October  2014  included  a  low- 
rate  initial  production  option,  and  the  Air  Force  had 
planned  to  award  a  separate  sole  source  contract 
for  full-rate  production.  Program  officials  stated  the 
full-rate  production  option  was  added  to  take 
advantage  of  the  increased  competition  resulting 
from  3DELRR's  participation  in  the  Defense 
Exportability  Features  Program,  which  provides 
contractors  an  early  opportunity  to  engage  in  DOD- 
sponsored  foreign  military  sales  and  associated 
logistics  support. 

Program  Office  Comments 

The  program  office  concurred  with  this  assessment 
and  provided  technical  comments,  which  were 
incorporated  where  appropriate. 


Page  158 


GAO-1 7-333SP  Assessments  of  Selected  Weapon  Programs 


Common  Name:  APT 

Advanced  Pilot  Trainer  (APT) 


With  its  Advanced  Pilot  Training  program  (APT),  the  Air 
Force  is  replacing  its  legacy  T-38C  trainer  fleet  and 
related  ground  equipment  by  developing  and  fielding 
newer,  more  technologically  advanced  trainer  aircraft 
and  an  associated  ground  based  training  system.  The 
APT  will  meet  the  Air  Force’s  advanced  fighter  pilot 
training  needs  and  close  training  gaps,  which  the 
T-38C  cannot  fully  address. 


Source:  U.S.  Air  Force 


Current  Status 

In  October  2009,  the  Air  Force  identified  a  gap  in  its  aircraft  training  capabilities  beginning  in  2018.  In  May 
2010,  the  Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics  approved  the  Air  Force’s 
plans  to  conduct  an  analysis  of  alternatives  for  closing  this  gap.  In  June  201 1 ,  the  analysis  of  alternatives 
recommended  that  the  existing  training  aircraft,  the  T-38C,  be  replaced  because  a  modification  program 
would  not  be  cost  effective,  nor  address  all  capability  gaps,  and  would  leave  the  Air  Force  with  aging  T-38C 
airframes. 

The  Air  Force  released  a  request  for  proposals  for  APT  in  December  2016  and  plans  to  award  a  fixed-price 
development  and  early  production  contract  for  by  December  2017.  According  to  the  program  office,  all 
required  technology  planned  for  the  aircraft  is  mature.  As  a  result,  the  OSD  approved  Acquisition  Strategy 
initiates  the  program  at  Milestone  B,  beginning  with  an  Engineering  and  Manufacturing  Development  (EMD) 
phase  limited  to  4  years.  The  Air  Force  plans  to  begin  development  on  the  APT  program  in  December  2017 
before  a  preliminary  design  review  is  held,  which  would  require  a  waiver  from  statutory  requirements,  and 
without  conducting  competitive  prototyping — actions  that  are  inconsistent  with  best  practices  for  product 
development.  Plowever,  according  to  Air  Force  officials,  all  perspective  vendors  will  offer  systems  which  are 
well  beyond  the  prototype  phase.  The  officials  also  maintain  APT’s  technical  risks  are  low  because  vendors 
must  provide  flight-test  data  from  aircraft  that  closely  match  the  offered  aircraft.  Because  the  APT  program 
does  not  yet  have  an  approved  acquisition  program  baseline,  the  estimated  development  cost  shown  below 
was  obtained  from  the  Air  Force’s  fiscal  year  2017  budget  submission.  The  Air  Force  expects  APT  to  achieve 
initial  operational  capability  in  the  fourth  quarter  of  fiscal  year  2024. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2017  dollars): 

Total  program:  TBD 

Research  and  development  (fiscal  years  2015  through  2021):  $836.7  million 
Procurement:  TBD 
Quantity:  350 

Next  Major  Program  Event:  Milestone  B  decision  planned  for  first  quarter  fiscal  year  2018 

Program  Office  Comments:  According  to  Air  Force  officials,  the  APT  acquisition  strategy  pursues 
capabilities  and  design  solutions  that  already  exist  in  the  marketplace.  Competitive  offerings  will  be  well 
beyond  prototype  and  preliminary  designs,  and  in  some  cases  offering  systems  that  are  currently  in 
production. 
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Common  name:  JSTARS  Recap 


Joint  Surveillance  Target  Attack  Radar  System  Recapitalization  (JSTARS  Recap) 


With  its  JSTARS  Recap  program,  the  Air  Force  is 
replacing  aging  JSTARS  aircraft — manned  Battle 
Management  Command  and  Control  (BMC2)  systems 
that  provide  surveillance  and  information  on  moving 
and  stationary  ground  targets — while  seeking  to  reduce 
weapon  system  operating  and  sustainment  costs, 
replacing  and  improving  JSTARS  capability,  and 
minimizing  development  and  integration  costs. 


Source:  U.S.  Air  Force 


Current  Status 

The  Air  Force’s  acquisition  strategy  for  JSTARS  Recap  is  to  competitively  procure  in-production,  business 
class  jets  and  equip  them  with  modern  search  radar,  BMC2,  and  broad-spectrum  communication 
subsystems.  To  reduce  technical  and  integration  risks,  as  well  as  future  upgrade  costs,  the  program  office 
plans  to  integrate  currently  available  systems  and  mature  technologies  using  open  system  architecture 
and  open  mission  systems,  in  accordance  with  the  design  architecture  developed  by  the  program  office. 

In  December  2015,  the  program  entered  the  technology  maturation  and  risk  reduction  phase.  The  program 
awarded  three  firm-fixed-price  contracts  to  Lockheed  Martin,  Northrop  Grumman,  and  Boeing  for  limited 
development  work  ahead  of  the  program’s  engineering  and  manufacturing  development  (EMD)  phase. 
Each  contractor  developed  a  preliminary  design  and  demonstrated  major  subsystem  prototypes.  Program 
officials  stated  the  preliminary  design  reviews  resulted  in  three  viable  designs.  Also  according  to  program 
officials,  within  the  technology  maturation  and  risk  reduction  phase,  the  program  is  conducting  radar  risk 
reduction  efforts  with  Northrop  Grumman  and  Raytheon  to  mature  radar  subsystem  technologies, 
demonstrate  prototypes,  further  reduce  integration  risk,  and  foster  competition. 

The  program  identified  seven  candidate  critical  technologies  and  is  currently  conducting  an  independent 
technology  readiness  assessment  (TRA)  before  EMD.  The  program  expects  to  complete  the  TRA  by  June 
2017  and  has  preliminarily  assessed  almost  all  candidate  critical  technologies  as  at  least  nearing  maturity. 
The  Air  Force  expects  JSTARS  Recap  to  achieve  initial  operational  capability  in  the  fourth  quarter  of  fiscal 
year  2024. 

Estimated  Total  Program  Cost  (fiscal  year  2017  dollars): 

Total  program:  $7,126.1  million 
Research  and  Development:  $3,039.8  million 
Procurement:  $4,086.3  million 
Total  Quantity:  17 

Next  Major  Program  Event:  Engineering  and  Manufacturing  Development  start,  Fiscal  Year  2018 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  program  office  provided 
technical  comments,  which  were  incorporated  where  deemed  appropriate. 
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Common  Name:  PAR 

Presidential  Aircraft  Recapitalization  (PAR) 


The  PAR  program  plans  to  replace  the 
current  VC-25A  presidential  aircraft  with  a 
new  modified  commercial  plane  to  transport 
the  President  of  the  United  States.  The 
PAR  aircraft  will  be  a  four  engine  wide- 
body,  commercial  derivative  aircraft, 
uniquely  modified  to  provide  the  President, 
staff,  and  guests  with  safe  and  reliable  air 
transportation  with  the  same  level  of 
security  and  communications  capability 
available  in  the  White  House. 


U.S.  AIR  FOItCl 

Source:  U,S.  Air  Forco. 


Current  Status 

DOD  approved  the  PAR  program’s  acquisition  strategy  in  September  2015  after  several  years 
of  acquisition  planning  and  studies  to  reduce  program  execution  risk.  In  September  2016,  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics  (USD  AT&L)  approved 
the  program  for  initiation  as  a  Major  Defense  Acquisition  Program  at  the  Engineering  and 
Manufacturing  Development  (EMD)  phase  and  authorized  the  release  of  a  Request  for 
Proposal  to  cover  efforts  in  the  EMD  phase,  which  will  result  in  modification  and  testing  of  two 
commerical  aircraft  to  be  fielded  to  meet  presidential  airlift  requirements.  Prior  to  entering  EMD, 
USD  AT&L  approved  waiving  several  requirements  related  to  affordability,  funding,  technology 
maturity,  and  preliminary  design  review.  USD  AT&L  determined  that  without  such  waivers, 

DOD  would  be  unable  to  meet  critical  national  security  objectives — specifically,  completing 
these  requirements  would  significantly  delay  the  acquisition  of  the  new  aircraft  and  replacement 
of  the  legacy  aircraft. 

The  PAR  program  continues  design  and  risk  reduction  studies,  which  may  assist  the  Air  Force 
in  making  additional  trade-offs  among  cost,  schedule  and  performance  objectives  prior  to 
establishing  the  Acquisition  Program  Baseline  and  determining  final  aircraft  quantities  in  2017. 
The  program  plans  to  modify  and  test  the  new  747-8  aircraft  in  a  phased  approach  starting  in 
2019,  using  research  and  development  funding.  Program  officials  acknowledge  technology 
integration  risks  but  stated  the  majority  of  the  required  mission-related  systems  are  currently 
operating  on  different  weapon  systems  and  have  legacy  or  related  equivalents  onboard  the 
existing  presidential  aircraft.  Once  development  is  complete,  the  aircraft  will  be  delivered  as 
fully  capable  to  support  presidential  missions,  planned  for  fiscal  year  2024. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2017  dollars): 

Total  program  (fiscal  years  2010-2021):  $3,074.0  million 
Research  and  development  (fiscal  years  2010-2021):  $2,759.9  million 
Procurement:  N/A 

Military  construction  (fiscal  years  2017-2019):  $314.1  million 
Quantity:  2 

Next  Major  Program  Event:  Start  of  preliminary  design  activities,  second  quarter  fiscal  year 
2017 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  program  office 
provided  technical  comments,  which  were  incorporated  where  deemed  appropriate. 
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Common  Name:  WSF-M 


Weather  System  Follow-on  -  Microwave  (WSF-M) 


The  Air  Force’s  WSF-M  is  expected  to  be  DOD’s 
next  weather  satellite  system.  WSF-M  is  intended  to 
contribute  to  a  family  of  space-based  environmental 
monitoring  systems  that  support  military  operations 
through  remote  sensing  of  weather  conditions,  such  as 
wind  speed  and  direction  at  the  ocean’s  surface  and 
space  weather,  and  by  providing  real-time  data  to  be 
used  in  models  for  weapon  system  planning  and 
weather  forecasting. 


Source:  U.S.  Air  Force. 


Current  Status 

WSF-M  is  DOD’s  third  effort  to  replace  the  aging  Defense  Meteorological  Satellite  Program,  following  two 
prior  cancellations.  The  Air  Force  is  developing  WSF-M  to  satisfy  3  of  1 1  requirements  that  DOD  examined 
in  an  analysis  of  alternatives  and  determined  were  mission  critical,  yet  insufficiently  met  by  other  sources. 
WSF-M  is  to  have  two  payloads — a  microwave  imager  to  collect  data  primarily  on  ocean  surface  vector  wind 
and  tropical  cyclone  intensity,  and  an  energetic  charged  particle  sensor  to  collect  space  weather  data — on  a 
polar-orbiting  satellite. 

As  a  precursor  to  WSF-M,  an  Operationally  Responsive  Space  technology  demonstration  program  is  to  use  an 
existing  microwave  sensor  from  NASA’s  Jet  Propulsion  Laboratory.  Combined  with  a  new  application  of 
existing  data,  the  demonstration  is  planned  to  partially  meet  ocean  surface  vector  wind  and  tropical  cyclone 
intensity  requirements,  supplement  current  operational  capability,  and  inform  WSF-M  technology 
development.  With  WSF-M,  the  Air  Force  plans  to  fully  meet  all  3  requirements  through  a  competitive  award 
for  up  to  two  satellites  and  by  hosting  an  energetic  charged  particle  sensor.  The  Air  Force  approved  the  WSF- 
M  acquisition  strategy  in  October  2016.  The  technology  demonstration  is  expected  to  be  ready  for  launch  in 
November  2017;  the  first  WSF-M  launch  is  expected  in  the  middle  of  fiscal  year  2023.  Both  are  to  be 
integrated  into  existing  ground  systems. 

According  to  Air  Force  plans,  WSF-M  is  expected  to  enter  development  with  one  critical  technology.  This 
critical  technology,  the  polarimetric  receiver,  is  expected  to  be  mature  by  development  start.  According  to  the 
Air  Force,  other  technologies  are  no  longer  considered  critical.  The  Air  Force  has  identified  software 
development  as  a  low  to  medium  risk  area. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2017  dollars): 

Total  program:  TBD 

Research  and  development  (fiscal  years  2012  through  2022):  $786.3  million 
Procurement:  TBD 

Quantity:  1  (technology  demonstration),  2  (development)  (includes  satellite  quantities  only) 

Next  Major  Program  Event:  System  preliminary  design  review,  second  quarter  2018 

Program  Office  Comment:  In  commenting  on  a  draft  of  this  assessment,  the  Air  Force  provided  technical 
comments,  which  were  incorporated  where  deemed  appropriate. 
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Joint  DOD-wide  Programs  Summary 


We  completed  individual  assessments  on  two  of  the  four  “joint”  or  DOD-wide  current  and  future  major  defense 
acquisition  programs — the  F-35  Lighting  II  and  Joint  Light  Tactical  Vehicle  (JLTV)  programs.  We  found  DOD 
currently  estimates  a  need  of  more  than  $233  billion  in  funding  to  complete  the  acquisition  of  these  two 
programs.  We  also  compared  these  two  programs’  first  full  estimates  of  cost  and  schedule  with  their  current 
estimates  and  found: 

•  net  cost  growth  exceeds  $100  billion,  all  of  which  occurred  more  than  5  years  ago  and  is  attributable  to  the 
F-35  Lightning  II  program,  and 

•  program  schedule  delays  average  approximately  40  months. 

Further,  the  F-35  program  completed  all  the  activities  associated  with  the  applicable  knowledge  based  best 
practices  we  assess,  although  they  were  not  fully  complete  at  the  time  the  knowledge  points  were  reached. 


Acquisition  Phase  and  Size  of  the  Currently  Estimated  Acquisition  Cost  for  Summary  of  Knowledge  Attained  to  Date  for 

Two  Programs  Assessed  the  Two  Programs  Assessed  Programs  Beyond  System  Development  Start 


Production 


System  development 
Technology  development 


•  Cost  growth  of  more  than  15  percent  and/or 
schedule  delays  of  more  than  6  months 

•  Cost  growth  of  1 5  percent  or  less  and  schedule 
delays  of  6  months  or  less 

No  first  full  estimate  available 


Note:  Bubble  size  is  based  on  each  program's  currently  estimated 
future  funding  needed. 


Fiscal  year  2017  dollars  in  billions 


JLTV 

(in  production) 


Knowledge 

Knowledge 

Knowledge 

Point  (KP)  1 

Point  2 

Point  3 

Resources 

Product 

Manufacturing 

and 

design 

processes  are 

requirements 

is  stable 

mature 

Program 

match 

common 

At  Current 

At  Current 

At 

Current 

name 

KP1  Status 

KP2  Status 

KP3 

Status 

F-35  JSF 

o  • 

o  • 

o 

• 

JLTV 

o  • 

■ 

■ 

■ 

o 

o 

o 

•  All  applicable  knowledge  practices 
completed 

OOne  or  more  applicable  knowledge 
practices  were  not  completed 


Cost  and  Schedule  Growth  on  Two  Programs 
in  the  Current  Portfolio 

Fiscal  year  2017  dollars  in  billions 


Knowledge  practice  is  not  applicable 

Information  not  available  for  knowledge 
practice 


Research  and  Procurement  Total 

development  costs  acquisition 

costs  costs 


□  Growth  from  first 
full  estimate 


|  |  First  full  estimate 


Average 
acquisition 
cycle  time 

(in  months) 


Note:  In  addition  to  research  and  development  and  procurement 
costs,  total  acquisition  cost  includes  acquisition  related  operation 
and  maintenance  and  system-specific  military  construction  costs. 


Source:  GAO  analysis  of  DOD  data.  |  GAO-1 7-333SP 
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Joint  DOD-wide  Program  Assessments 

2-page  assessments 

Page  number 

F-35  Lightning  II  Program  (F-35) 

165 

Joint  Light  Tactical  Vehicle  (JLTV) 

167 
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Lead  Component:  DOD 


Common  Name:  F-35 


F-35  Lightning  II  Program  (F-35) 

DOD  is  developing  and  fielding  a  family  of  next 
generation  strike  fighter  aircraft,  integrating  stealth 
technologies  with  advanced  sensors  and  computer 
networking  capabilities  for  the  United  States  Air 
Force,  Marine  Corps,  and  Navy  as  well  as  eight 
international  partners.  The  Air  Force's  F-35A  is 
expected  to  replace  the  air-to-ground  attack 
capabilities  of  the  F-1 6  and  A-1 0  and  complement 
the  F-22A.  The  F-35B  variant  is  expected  to  replace 
the  Marine  Corps'  F/A-18  and  AV-8B  aircraft.  The  F- 
35C  will  complement  the  Navy's  F/A-18E/F. 


Source:  ©2016  Lockheed  Martin. 


sncept 


System  development 


Production 


▲  ▲ 

Program  Development 
start  start 

(11/96)  (10/01) 


AAA 
Design  Production  Milestone 
review  decision  recertification 
(2/06  &  6/07)  (6/07)  (3/12) 


A  A  A  A  A 

USMC  initial  USAF  initial  GAO  Start  USN  initial 
capability  capability  review  operational  test  capability 
(7/15)  (8/16)  (1/17)  (8/17)  (8/18) 


Program  Essentials 

Program  Performance  (fiscal  year  2017 

dollars  in  millions) 

Prime  contractors:  Lockheed  Martin, 

As  of 

Latest 

Percent 

Pratt  &  Whitney 

10/2001 

12/2015 

change 

Program  office:  Arlington,  VA 

Research  and  development  cost 

$42,277.5 

$63,838.6 

51 .0% 

Funding  needed  to  complete: 

Procurement  cost 

$187,565.4 

$267,912.7 

42.8% 

R&D:  $1,706.7  million 

Total  program  cost 

231,806.3 

$336,152.4 

45.0% 

Procurement:  $209,658.2  million 

Program  unit  cost 

$80,881 

$136,814 

69.2% 

Total  funding:  $213,918.1  million 

Total  quantities 

2,866 

2,457 

-14.3% 

Procurement  quantity:  2,158 

Acquisition  cycle  time  (months) 

175 

237 

35.4% 

All  of  the  F-35  program’s  critical  technologies  are 
fully  mature.  A  previous  critical  technology  that 
had  not  been  fully  matured  has  now  been 
deferred  to  follow-on  development.  Although  the 
aircraft  designs  were  not  stable  at  their  critical 
design  reviews  in  2006  and  2007,  all  baseline 
engineering  drawings  have  since  been  released. 
While  developmental  testing  is  progressing, 
design  changes  are  likely  until  the  program 
completes  these  tests.  Manufacturing  efficiency  is 
steady,  and  program  officials  stated  that  critical 
production  processes  are  in  control.  The  program 
is  planning  follow-on  modernization  to  address 
capability  deferrals,  upgrade  existing  capabilities, 
and  integrate  additional  weapons  into  the  F-35 
aircraft. 
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Attainment  of  Product  Knowledge 


As  of  January  2017 


Resources  and  requirements  match 


Demonstrate  all  critical  technologies  are  very  close  to 
final  form,  fit,  and  function  within  a  relevant  environment 


STATUS  CURRENT 
AT  STATUS 


DEVELOPMENT 

START 


O 


Demonstrate  all  critical  technologies  in  form,  fit  and 
function  within  a  realistic  environment 


o 

o 


Complete  a  system-level  preliminary  design  review 


Product  design  is  stable 


Release  at  least  90  percent  of  design  drawings 
Test  a  system-level  integrated  prototype 


DESIGN  REVIEW 


o 

o 


Manufacturing  processes  are  mature 


Demonstrate  Manufacturing  Readiness  Level  of  at  least 
9  or  critical  processes  are  in  statistical  control 


PRODUCTION 

START 


O 


Demonstrate  critical  processes  on  a  pilot  production  line  o 

Test  a  production-representative  prototype  o 


9  Knowledge  attained 
O  Knowledge  not  attained 


Information  not  available 
Not  applicable 


Common  Name:  F-35 


F-35  Program 

Technology  and  Design  Maturity 

All  of  the  F-35  program’s  critical  technologies  are 
fully  mature.  In  201 3,  the  program  deferred  plans  for 
integrating  a  prognostics  and  health  management 
system  technology  into  the  aircraft  design.  The 
program  had  identified  this  technology,  which  is  part 
of  the  F-35  Autonomic  Logistics  Information  System 
(ALIS),  as  critical  but  not  fully  mature.  The  program 
now  plans  to  integrate  this  technology  into  the  F-35 
design  as  part  of  a  later  block  of  aircraft.  In  addition, 
the  program  has  yet  to  incorporate  engine  health 
data  into  ALIS.  Further,  the  program  continues  to 
address  issues  with  other  technologies  including  the 
next-generation  helmet,  which  is  currently  providing 
degraded  night  vision  capabilities  at  sea. 

Although  the  aircraft  designs  were  not  stable  at  their 
critical  design  reviews  in  2006  and  2007,  the 
program  has  since  released  all  baseline 
engineering  drawings.  As  the  program  nears  the 
end  of  developmental  testing,  officials  continue  to 
identify  and  address  technical  risks.  For  example, 
the  prime  contractor  implemented  design  changes 
to  address  deficiencies  with  the  arresting  hook  and 
engine  and,  during  2016,  began  addressing  ejection 
seat  problems.  In  addition,  the  Navy  recently 
identified  problems  affecting  the  F-35C  wing 
structure  when  carrying  an  AIM-9X  missile  and  is 
identifying  design  changes.  The  program  faces  the 
risk  of  further  design  changes  until  developmental 
testing  is  complete. 

Production  Maturity 

Aircraft  manufacturing  deliveries  remain  steady  and 
totaled  199  production  aircraft  as  of  December 
2016.  Since  the  start  of  production,  the  contractor's 
production  processes  have  continued  to  mature, 
and  program  officials  stated  that  critical  production 
processes  are  now  in  control.  To  enable  continued 
improvements  and  to  increase  quality,  the 
contractor  tracks  process  control  data  and  other 
quality  indicators.  Production  part  shortages  remain 
a  risk  as  suppliers  face  additional  pressures  of 
balancing  an  increased  production  rate  amid  the 
simultaneous  need  to  sustain  a  growing  operational 
fleet. 

Other  Program  Issues 

In  July  2015,  the  Marine  Corps  declared  F-35B 
initial  operational  capability  (IOC)  prior  to  meeting 
all  expected  requirements.  In  particular,  F-35B 
offered  only  limited  capabilities  with  respect  to  eight 


performance  requirements,  including  those 
centered  on  sensor  fusion,  electronic  warfare,  and 
communication.  The  full  capabilities  for  these 
requirements  were  delivered  in  a  subsequent 
software  upgrade.  The  Air  Force  declared  F-35A 
IOC  in  August  2016  having  met  and  delivered  all 
expected  capabilities.  The  Navy  F-35C  IOC  date  is 
scheduled  for  August  2018. 

F-35  flight  testing  has  been  consistently  delayed 
over  the  past  several  years,  and  in  the  past  year,  the 
program  encountered  additional  delays.  The 
program  estimates  it  needs  an  additional  5  months 
to  complete  developmental  testing  and  $451  million 
to  complete  the  development  program.  These 
estimates  are  optimistic  and  assume  that  the 
program  will  be  able  to  accomplish  remaining  tests 
at  a  significantly  faster  pace  than  historically 
demonstrated.  The  Director,  Operational  Test  and 
Evaluation  reported  that  the  program  will  need 
significantly  more  time  and  money  than  it  now 
forecasts,  which  could  jeopardize  the  program's 
schedule  for  initial  operational  test  and  evaluation, 
full-rate  production,  and  follow-on  modernization. 

Program  Office  Comments 

According  to  program  officials,  the  prognostics  and 
health  management  system,  less  the  downlink 
capability,  has  been  operating  on  the  aircraft  and 
continues  to  mature.  The  prognostics  and  health 
management  system  downlink  capability  was 
deferred  due  to  certification  concerns,  not  immature 
technology.  Currently,  officials  state  engine 
maintenance  data  is  processed  using  a  dedicated 
engine  tool.  Engine  data  will  be  processed  in  ALIS 
with  the  fielding  of  ALIS  2.0.2  in  spring  2017,  and 
the  update  will  eliminate  the  need  for  the  engine  tool 
workaround.  Officials  also  said  the  night  vision 
capability  of  the  helmet  has  been  significantly 
improved  with  the  newest  generation  helmet.  F-35s 
are  now  permanently  operating  overseas.  In 
January  2017,  the  Marine  Corps  deployed  10  F- 
35Bs  from  its  air  station  in  Yuma,  Arizona  to  the  air 
station  in  Iwakuni,  Japan.  Program  officials  noted 
some  risk  exists  to  completion  of  system 
development;  however,  steady  reduction  of  known 
challenges  and  a  slowly  decreasing  rate  of  new 
discovery  should  allow  for  completion  within  the 
established  acquisition  program  baseline. 
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Lead  Component:  DOD 


Common  Name:  JLTV 


Joint  Light  Tactical  Vehicle  (JLTV) 


The  Army  and  Marine  Corps'  JLTV  is  a  family  of 
vehicles  being  developed  to  replace  the  High 
Mobility  Multipurpose  Wheeled  Vehicle  (HMMWV) 
for  some  missions.  The  JLTV  is  expected  to  provide 
protection  for  passengers  against  current  and  future 
battlefield  threats,  increased  payload  capacity,  and 
improved  automotive  performance  over  the  up- 
armored  HMMWV.  It  will  also  be  transportable  by  air 
and  ship.  Two-  and  four-seat  variants  are  planned 
with  multiple  mission  configurations. 


Source:  U.S.  Army. 


^  Concept 


System  development 


Productic 


A 

A 

A 

A 

A 

A 

A 

A 

P  rogram  start 

Development 

Design 

Low-rate 

GAO 

Begin 

Full-rate 

Initial 

(12/07) 

start 

review 

decision 

review 

operational  test 

decision 

capability 

(8/12) 

(1/13) 

(8/15) 

(1/17) 

(2/18) 

(12/18) 

(12/19) 

Program  Performance  (fiscal  year  2017  dollars  in  millions) 


Program  Essentials 

Prime  contractor:  Oshkosh  Defense 
LLC 

Program  office:  Harrison  Township,  Ml 
Funding  needed  to  complete: 

R&D:  $122.0  million 
Procurement:  $19,010.8  million 
Total  funding:  $19,132.8  million 
Procurement  quantity:  53,372 


Research  and  development  cost 
Procurement  cost 
Total  program  cost 
Program  unit  cost 
Total  quantities 

Acquisition  cycle  time  (months) 


As  of 

Latest 

Percent 

10/2012 

12/2015 

change 

$1,032.6 

$978.2 

-5.3% 

$23,373.8 

$19,489.0 

-16.6% 

$24,444.9 

$20,467.2 

-1 6.3% 

$0,447 

$0,374 

-1 6.2% 

54,730 

54,714 

0.0% 

125 

144 

-29.6% 

Both  JLTV  critical  technologies — underbelly 
protection  armor  and  side-kit  armor — are  mature 
and,  according  to  officials,  have  been  integrated 
and  tested  on  production-representative  vehicles. 

During  development,  operational  testers  found 
shortcomings  that  can  likely  only  be  overcome 
with  changes  in  tactics  or  procedures.  The 
government  conducted  design  understanding 
reviews,  instead  of  a  formal  critical  design  review, 
for  three  competing  vehicle  designs  to  assess 
their  technical  baselines.  To  assess 
manufacturing  readiness  and  risk  prior  to 
production  start,  the  program  conducted  a 
manufacturing  readiness  assessment  using 
manufacturing  readiness  levels.  While  the 
program’s  manufacturing  process  maturity  may 
have  reached  DOD's  recommended  level  for 
production,  it  has  not  reached  a  level  that 
indicates  processes  are  in  control,  as 
recommended  by  best  practices. 
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Demonstrate  Manufacturing  Readiness  Level  of  at  least 
9  or  critical  processes  are  in  statistical  control 


PRODUCTION 

START 


o  o 


Demonstrate  critical  processes  on  a  pilot  production  line 
Test  a  production-representative  prototype 


^  Knowledge  attained 
o  Knowledge  not  attained 


Information  not  available 
Not  applicable 


Common  Name:  JLTV 


JLTV  Program 

Technology  and  Design  Maturity 

According  to  the  program  office,  JLTV's  two  critical 
technologies — underbelly  protection  armor  and 
side-kit  armor — are  fully  mature.  According  to  Army 
officials,  prototype  systems  with  the  critical 
technologies  have  been  tested  in  a  realistic 
environment.  The  JLTV  program  office  declared 
both  technologies  are  mature  and  have  been 
demonstrated  under  operational  conditions. 

The  program  office  did  not  hold  a  formal  critical 
design  review  during  development,  but  instead 
conducted  design  understanding  reviews  with 
contractors  between  December  2012  and  January 
2013.  According  to  program  officials,  these  reviews 
were  at  a  level  of  detail  similar  to  a  critical  design 
review  and  verified  that  all  contractors  had  more 
than  90  percent  of  the  design  files  under 
configuration  control.  According  to  operational 
testers,  the  Oshkosh  JLTV  design,  which  the  Army 
ultimately  selected  for  production,  more  than 
doubles  the  reliability  of  the  up-armored  HMMWV, 
as  measured  by  average  miles  traveled  before 
mechanical  failure.  Testers  also  noted  Army  units 
cannot  accomplish  air  assault  missions  with  JLTVs 
that  already  have  add-on  armor  installed  because 
the  combined  weight  exceeds  the  CH-47F 
helicopter's  lift  capacity.  However,  program  officials 
stated  the  JLTV  requirement  for  CH-47F  air  assault 
missions  is  with  base  armor  protection  and  not  with 
the  add-on  armor  installed.  Operational  testers  also 
noted  the  JLTV's  design  did  not  offer  sufficient 
capability  to  carry  mission  equipment,  supplies,  or 
water  for  more  than  a  single  day,  which  limits  the 
vehicle's  mission  type  and  duration.  This  limitation 
may  require  operational  commanders  to  manage 
mission  payloads  to  remain  within  JLTV's 
capacities. 

Production  Maturity 

The  Army  did  not  demonstrate  production  readiness 
with  statistical  process  control  data  prior  to 
awarding  the  production  contract  in  August  2015. 
For  the  engineering  and  manufacturing 
development  phase,  Oshkosh  built  20  prototype 
JLTV  vehicles  on  an  assembly  line  used  to  produce 
the  Family  of  Medium  Tactical  Vehicles  and  a 
version  of  the  Mine-Resistant  Ambush  Protected 
vehicle.  According  to  program  officials,  this  quantity 
was  insufficient  for  providing  meaningful  statistical 


control  data.  To  assess  production  readiness  and 
manufacturing  risks  prior  to  production  start,  the 
program  conducted  a  manufacturing  readiness 
assessment  using  manufacturing  readiness  levels. 
According  to  program  officials,  the  contractor  has 
now  begun  collecting  production  process  capability 
index  data  from  low-rate  production  units  for 
statistical  control  purposes.  Also,  the  design  and 
manufacturing  teams  continue  identifying  key 
product  characteristics  and  manufacturing 
processes.  The  program  and  the  contractor  are 
working  together  to  identify  critical  and  significant 
characteristics  for  each  subsystem,  a  process  the 
program  office  says  should  take  a  year.  While 
program  officials  declared  manufacturing  process 
maturity  reached  DOD's  recommended  level  for 
production,  the  program  has  not  yet  demonstrated 
that  JLTV  manufacturing  processes  are  in  statistical 
control  as  recommended  by  best  practices. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  technical  comments,  which 
were  incorporated  where  deemed  appropriate. 
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Agency  Comments 


We  provided  a  draft  of  this  report  to  DOD  for  comment.  In  its  comments, 
reproduced  in  appendix  VII,  DOD  generally  concurred  with  our 
observations.  DOD  also  provided  us  with  technical  comments,  which  we 
incorporated  as  appropriate. 

In  its  comments,  DOD  noted  that  our  report  identifies  many  ways  in  which 
the  department  continues  to  drive  down  the  cost  of  the  acquisition 
portfolio.  Further,  DOD  stated  that  our  observations  appear  to  validate  its 
focus  on  continuous  improvement  on  cost,  schedule,  and  performance 
measures  of  programs  using  the  “Better  Buying  Power”  and  other 
initiatives.  The  department  also  identified  plans  to  publish  its  own 
comprehensive  report  on  the  acquisition  system  in  the  summer  of  2017, 
as  it  has  done  in  previous  years. 


We  are  sending  copies  of  this  report  to  interested  congressional 
committees  and  offices;  the  Secretary  of  Defense;  the  Secretaries  of  the 
Army,  Navy,  and  Air  Force;  and  the  Director  of  the  Office  of  Management 
and  Budget.  In  addition,  the  report  will  be  made  available  at  no  charge  on 
the  GAO  Web  site  at  http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  concerning  this  report,  please 
contact  me  at  (202)  512-4841 .  Contact  points  for  our  offices  of 
Congressional  Relations  and  Public  Affairs  may  be  found  on  the  last  page 
of  this  report.  Staff  members  making  key  contributions  to  this  report  are 
listed  in  appendix  VIII. 


Michael  J.  Sullivan 

Director,  Acquisition  and  Sourcing  Management 
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List  of  Committees 


The  Honorable  John  McCain 
Chairman 

The  Honorable  Jack  Reed 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Thad  Cochran 
Chairman 

The  Honorable  Richard  J.  Durbin 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Mac  Thornberry 
Chairman 

The  Honorable  Adam  Smith 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  Kay  Granger 
Chairwoman 

The  Honorable  Pete  Visclosky 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
House  of  Representatives 
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Appendix  I:  Scope  and  Methodology 


Analysis  Of  the  Cost  To  develop  our  1 1  observations  on  the  cost  and  schedule  of  the 

Performance  of  DOD’S  Department  of  Defense  portfolio  of  current  major  defense  acquisition 

Portfolio  of  Maior  Defense  programs,  we  obtained  and  analyzed  cost,  quantity,  and  schedule  data 
.  ...  _  ^  from  Selected  Acquisition  Reports  (SAR)  and  other  information  in  the 

Acquisition  rrograms  Defense  Acquisition  Management  Information  Retrieval  Purview  system, 

referred  to  as  DAMIR.1  We  entered  this  data  into  a  database  and  verified 
that  the  data  were  entered  correctly.  We  converted  all  cost  information  to 
fiscal  year  2017  dollars  using  conversion  factors  from  the  DOD 
Comptroller’s  National  Defense  Budget  Estimates  for  Fiscal  Year  2017 
(table  5-9).  To  assess  the  reliability  of  the  SAR  data,  we  reviewed  our 
previous  assessment  and  DOD  officials’  responses  regarding  any 
changes  to  DAMIR’s  data  quality  control  procedures.  We  determined  that 
the  SAR  data  and  the  information  retrieved  from  DAMIR  were  sufficiently 
reliable  for  the  purposes  of  this  report.  Our  assessment  includes 
comparisons  of  cost  and  schedule  changes  over  the  past  year,  5  years, 
and  from  baseline  estimates  that  utilize  SAR  data  from  December  2015, 
December  2014,  December  2010,  and  from  the  programs’  initial  SAR 
submissions.  We  also  analyzed  the  data  to  determine  the  number  of 
programs  in  each  portfolio  year.  In  general,  we  refer  to  the  78  major 
defense  acquisition  programs  with  SARs  dated  December  2015  as  DOD’s 
201 6  or  current  portfolio  and  use  a  similar  convention  for  prior  year 
portfolios.  We  retrieved  data  on  research,  development,  test,  and 
evaluation;  procurement;  military  construction,  acquisition  operation  and 
maintenance,  and  total  acquisition  costs,  as  well  as  schedule  estimates 
for  the  78  programs  in  the  201 6  portfolio.2 

We  divided  2  programs  into  two  distinct  elements,  because  DOD  reports 
performance  data  on  them  separately.  As  a  result  some  of  our  analysis 
reflects  a  total  of  80  programs  and  sub-elements.  The  Missile  Defense 
Agency’s  Ballistic  Missile  Defense  System  and  its  elements  are  excluded 
from  all  analyses  as  they  do  not  have  an  integrated  long-term  baseline, 
which  prevents  us  from  assessing  the  program’s  cost  progress  or 
comparing  it  to  other  major  defense  acquisition  programs. 


1  DAMIR  Purview  is  an  executive  information  system  operated  by  the  Office  of  the  Under 
Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics  Acquisition  Resources  and 
Analysis. 

2  We  refer  to  research,  development,  test,  and  evaluation  costs  as  research  and 
development  or  simply  as  development  costs  in  this  report.  Total  acquisition  cost  includes 
research  and  development  and  procurement  costs  as  well  as  acquisition  related  operation 
and  maintenance  and  system-specific  military  construction  costs. 
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Appendix  I:  Scope  and  Methodology 


For  our  first  observation,  we  compared  the  2016  portfolio  with  the 
programs  that  issued  SARs  in  December  2014  (2015  portfolio)  to  identify 
the  programs  that  exited  and  entered  the  current  portfolio  and  the  total 
cost  and  schedule  change  in  the  current  portfolio  over  the  past  year. 

We  then  divided  the  programs  into  percent  cost  change  categories  based 
on  the  percent  change  in  total  acquisition  cost  they  experienced  over  the 
past  year.  We  then  totaled  the  number  of  programs  in  each  category  and 
the  total  cost  change  of  the  programs  in  each  category. 

For  the  second  observation,  we  identified  the  development  and 
procurement  funding  and  the  change  in  these  over  the  past  year.  We 
identified  the  specific  programs  with  the  largest  development  and 
procurement  cost  increases  and  whether  those  were  due  to  quantity 
changes. 

For  the  third  observation,  we  aggregated  funding  stream  data  for  the  total 
planned  investment  of  each  portfolio  from  DAMIR  for  each  year  since 
2006  to  determine  any  trends.  We  determined  the  yearly  totals  for 
research  and  development,  procurement,  and  total  acquisition  cost.  To 
distinguish  the  funding  already  invested  from  the  funding  remaining  that  is 
needed  to  complete  the  programs  in  each  portfolio  since  2006,  we  used 
funding  stream  data  obtained  from  DAMIR  for  each  December  SAR 
submission  for  the  years  2005  (2006  portfolio)  through  2015  (2016 
portfolio).  We  define  funding  invested  as  all  funding  that  has  been 
provided  to  the  programs  in  the  fiscal  year  of  the  annual  SAR  submission 
(this  includes  fiscal  year  2016  for  the  December  2015  submission)  and 
earlier,  while  funding  remaining  is  all  the  amounts  that  will  be  provided  in 
the  fiscal  years  after  the  annual  SAR  submission  (fiscal  year  2017  and 
later  for  the  December  2015  submission). 

For  our  fourth  observation,  we  determined  the  cost  and  schedule  changes 
on  defense  acquisition  programs  in  the  current  portfolio  over  the  past  5 
years  and  from  baseline  estimates.  To  do  this,  we  collected  data  from 
December  2015,  December  2010,  and  from  programs’  initial  SARs; 
acquisition  program  baselines;  and  program  offices.  In  addition,  we 
analyzed  programs’  cost  growth  after  three  key  decision  points: 
development  start,  critical  design  review,  and  production  decision. 

For  programs  less  than  a  year  old,  we  calculated  the  difference  between 
the  December  2015  SAR  current  estimate  and  the  first  full  estimate  in 
order  to  identify  the  cost  and  schedule  change  over  the  past  year.  For 
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programs  less  than  5  years  old,  we  took  a  similar  approach  when 
calculating  the  cost  and  schedule  change  over  the  past  5  years. 

For  our  fifth  observation,  in  order  to  determine  whether  programs 
experienced  an  increase  or  decrease  in  buying  power  over  the  past  year, 
we  used  data  on  the  programs’  number  of  procurement  units, 
procurement  cost  changes,  and  average  procurement  unit  costs. 

GAO  calculated  cost  change  “due”  to  quantity  changes  as  the  change  in 
quantity  over  the  last  year  multiplied  by  the  average  procurement  unit 
cost  for  the  program  a  year  ago.  GAO  calculated  cost  change  “not  due”  to 
quantity  changes  as  the  current  acquisition  quantity  times  the  change  in 
average  per  unit  costs.  In  practice,  changes  in  quantity  will  often  affect 
per  unit  cost — as  discussed  in  this  appendix — so  this  is  more  precisely 
described  as  “Cost  change  due  to  change  in  quantity  assuming  no 
change  in  average  procurement  unit  cost”  and  “Cost  change  due  to 
change  in  average  procurement  unit  cost.”  If  changes  in  quantity  affect 
per  unit  cost,  those  changes  will  appear  in  the  cost  change  “not  due”  to 
quantity  changes. 

The  resulting  dollar  amount  is  considered  a  change  due  solely  to  shifts  in 
the  number  of  units  procured  and  may  overestimate  the  amount  of 
change  expected  when  quantities  increase  and  underestimate  the 
expected  change  when  quantities  decrease,  as  it  does  not  account  for 
other  effects  of  quantity  changes  on  procurement  such  as  gain  or  loss  of 
learning  in  production  that  could  result  in  changes  to  unit  cost  over  time  or 
the  use  or  absence  of  economic  orders  of  material.  However,  these 
changes  are  accounted  for  as  part  of  the  change  in  cost  not  due  to 
quantities. 

For  our  sixth  observation,  we  evaluated  program  performance  against 
high-risk  criteria  discussed  by  DOD,  the  Office  of  Management  and 
Budget,  and  GAO.  We  calculated  how  many  programs  had  less  than  a  2 
percent  increase  in  total  acquisition  cost  over  the  past  year,  less  than  a 
10  percent  increase  over  the  past  5  years,  and  less  than  a  15  percent 
increase  from  baseline  estimates  using  data  from  SARs,  initial  acquisition 
program  baselines,  and  program  offices.  We  calculated  the  percentage  of 
programs  meeting  each  of  these  high-risk  criteria  for  the  201 2-201 6  time 
frame  to  identify  any  changes. 

For  programs  with  multiple  sub-programs  presented  in  the  SARs,  we 
calculated  the  net  effect  of  the  sub-programs  to  reach  an  aggregate 
program  result. 
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For  our  seventh  observation,  we  determined  which  programs  had  a  first 
full  estimate — or  started  system  development — after  the  implementation 
date  of  Dec.  4,  2009  for  the  Weapon  Systems  Acquisition  Reform  Act  of 
2009  (WSARA).  We  then  examined  these  programs  cost  and  schedule 
changes  over  the  past  year  as  a  group. 

For  our  eighth  observation,  we  determined  what  percent  of  the  portfolio’s 
total  acquisition  cost  was  allocated  to  these  programs. 

For  the  ninth  observation,  we  divided  these  programs  into  percent  cost 
change  categories  based  on  the  percent  change  in  total  acquisition  cost 
they  experienced  over  the  past  year.  We  then  totaled  the  number  of 
programs  in  each  category  and  the  total  cost  change  of  the  programs  in 
each  category. 

For  our  10th  observation,  we  identified  programs  initial  operational 
capability  and  start  and  end  dates  of  their  operational  testing.  We 
determined  whether  the  initial  operational  capability  date  was  before, 
during,  or  after  its  testing  dates.  Based  on  our  determination,  we  summed 
and  analyzed  what  percent  of  programs  were  in  each  category.  For  some 
programs,  either  one  or  both  of  these  dates  were  not  available. 

For  the  1 1th  observation,  we  used  SAR  data  to  determine  which  25 
programs  had  the  largest  total  acquisition  cost,  and  the  prime  contractors 
associated  with  these  programs.  We  gathered  information  from 
Bloomberg  on  the  equity  prices  of  each  of  these  contractors  from  about 
1980  to  2016  from  these  equities  to  the  S&P  500®  and  the  Industrials 
sector,  as  defined  by  the  Standard  and  Poor’s  Global  Industry 
Classification  Standard. 


Analysis  Of  Acquisition  To  develop  observations  on  how  DOD  is  implementing  acquisition 

Initiatives  reforms,  we  reviewed  the  DOD  Instruction  5000.02,  the  Weapon  Systems 

Acquisition  Reform  Act  of  2009  (WSARA),  and  the  September  19,  2014, 
Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics 
“Better  Buying  Power  3.0  Interim  Release”  as  well  as  earlier  related 
memoranda.3  We  analyzed  questionnaire  data  received  from  the  45 
current  and  9  future  major  defense  acquisition  programs  in  our 
assessment  to  determine  the  extent  to  which  acquisition  reforms  have 


3Pub.  L.  No.  111-23. 
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been  implemented.  We  assessed  the  P-8A  Poseidon  Multi-Mission 
Maritime  Aircraft  Increment  3  as  part  of  the  9  future  major  defense 
acquisition  programs  not  yet  in  the  portfolio.  During  the  course  of  our 
review,  we  learned  that  DOD  no  longer  plans  to  manage  Increment  3 
separately  from  the  existing  P-8A  program.  We  will  reflect  this  change  in 
future  assessments  of  the  P-8A  program.  We  determined  which  programs 
have  established  affordability  constraints  and,  for  current  programs, 
examined  the  average  development  cost  growth  on  programs  with  these 
constraints  compared  to  those  without.  We  tallied  programs  that 
conducted  “should-cost”  analyses  and  identified  realized  and/or  future 
potential  savings.  We  also  analyzed  whether  programs  are  planning  for 
competition  throughout  the  acquisition  life-cycle  and  their  plans  to 
evaluate  cybersecurity  vulnerabilities  by  2019. 


Analysis  of  Selected  DOD 
Programs  Using 
Knowledge-Based  Criteria 
and  Program  Concurrency 


To  collect  data  from  current  and  future  major  defense  acquisition 
programs — including  cost  and  schedule  estimates,  technology  maturity, 
and  planned  implementation  of  acquisition  reforms — we  distributed  one 
data  collection  instrument  and  two  electronic  questionnaires,  one 
questionnaire  for  the  45  current  programs  and  a  slightly  different 
questionnaire  for  the  9  future  programs.  Both  of  the  questionnaires  were 
web-based  so  that  respondents  could  respond  and  submit  their  answers 
online.  We  received  responses  from  all  of  the  programs  we  assessed 
from  August  to  October  2016.  To  ensure  the  reliability  of  the  data 
collected  through  the  data  collection  instrument  and  our  questionnaires, 
we  took  a  number  of  steps  to  reduce  measurement  error  and  non¬ 
response  error. 


These  steps  included:  conducting  three  pretests  of  the  future  major 
defense  acquisition  program  questionnaire  and  three  pretests  for  the 
current  major  defense  acquisition  program  questionnaire  prior  to 
distribution  to  ensure  that  our  questions  were  clear,  unbiased,  and 
consistently  interpreted;  reviewing  responses  to  identify  obvious  errors  or 
inconsistencies;  cross-referencing  information  provided  in  the  data 
collection  instrument  with  the  questionnaire;  and  conducting  follow-up  to 
clarify  responses  when  needed.  Our  pretests  covered  each  branch  of  the 
military  to  better  ensure  that  the  questionnaires  could  be  understood  by 
officials  within  each  branch. 


Our  analysis  of  how  well  programs  are  adhering  to  a  knowledge-based 
acquisition  approach  focuses  on  45  major  defense  acquisition  programs 
that  are  mostly  in  development  or  the  early  stages  of  production.  To 
assess  the  knowledge  attained  by  key  decision  points  (system 
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development  start  or  detailed  design  contract  award  for  shipbuilding 
programs,  critical  design  review  or  fabrication  start  for  shipbuilding 
programs,  and  production  start),  we  collected  data  from  program  offices 
about  their  knowledge  at  each  point.4  In  particular,  we  focused  on  the  1 1 
programs  that  crossed  these  key  acquisition  points  in  2016  or  planned  to 
in  early  2017  and  evaluated  their  adherence  to  knowledge-based 
practices. 

We  also  provide  information  on  how  much  knowledge  is  obtained  at  key 
decision  points  by  programs  which  accomplished  these  previously.  We 
also  included  observations  on  the  knowledge  that  9  future  programs 
expect  to  obtain  before  starting  development.  We  did  not  validate  the  data 
provided  by  the  program  offices,  but  reviewed  the  data  and  performed 
various  checks  to  determine  that  they  were  reliable  enough  for  our 
purposes.  Where  we  discovered  discrepancies,  we  clarified  the  data 
accordingly. 

For  programs  that  have  passed  a  key  decision  point  and  have  since  been 
restructured,  we  will  continue  to  assess  them  against  their  original  cost 
and  schedule  estimates  at  that  milestone  or  decision  point,  such  as 
development  start.  We  will  not  reassess  a  program  at  milestones  that 
have  already  been  reached  if  a  program  is  repeating  a  key  decision  point 
or  milestone  such  as  milestone  B.  We  will  keep  our  original  assessment 
of  the  program’s  knowledge  attained  at  the  original  milestone.  However, 
we  will  change  a  future  milestone  date  if  that  milestone  had  not  yet  been 
reached  and  assess  the  program  for  its  implementation  of  our  best 
practices  at  that  point  in  time. 

The  54  current  and  future  programs  included  in  our  assessment  were  in 
various  stages  of  the  acquisition  cycle,  and  not  all  of  the  programs 
provided  information  on  knowledge  obtained  at  each  point.  Programs 
were  not  included  in  our  assessments  at  key  decision  points  if  relevant 
data  were  not  available.  Our  analysis  of  knowledge  attained  at  each  key 
point  includes  factors  that  we  have  previously  identified  as  being  key  to  a 


4Analysis  of  shipbuilding  programs  for  technology  maturity  and  design  maturity  differ  as 
shipbuilding  programs  must  meet  these  metrics  at  different  points  in  the  acquisition  cycle 
according  to  GAO’s  methodology  for  ships  (see  GAO,  Best  Practices:  High  Levels  of 
Knowledge  at  Key  Points  Differentiate  Commercial  Shipbuilding  from  Navy  Shipbuilding, 
GAO-09-322  (Washington,  D.C.:  May,  13,  2009).  As  a  result  best  practices  for  a 
knowledge  based  acquisition  approach  differ  for  shipbuilding  programs.  For  these 
reasons,  we  exclude  the  six  shipbuilding  programs  in  DOD’s  portfolio  from  parts  of  our 
analysis  at  each  of  the  three  key  decision  points. 
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knowledge-based  acquisition  approach,  including  holding  early  systems 
engineering  reviews,  testing  an  integrated  prototype  prior  to  the  design 
review,  using  a  reliability  growth  curve,  planning  for  manufacturing,  and 
testing  a  production-representative  prototype  prior  to  the  making  a 
production  decision.  Additional  information  on  how  we  collect  these  data 
is  found  in  the  product  knowledge  assessment  section  of  this  appendix. 
See  also  appendix  IV  for  a  list  of  the  practices  that  are  associated  with  a 
knowledge-based  acquisition  approach. 

To  assess  program  development  testing  and  production  concurrency,  we 
identified  the  programs — among  those  we  included  in  our  assessment — 
with  production  start  dates.  We  used  the  questionnaire  responses  from 
those  programs  to  identify  the  dates  for  the  start  and  end  of 
developmental  testing,  compared  those  dates  to  the  timing  of  each 
program’s  production  decision  and  determined  the  number  of  months,  if 
any,  of  developmental  testing  done  after  production  start.  We  then 
compared  the  number  of  overlapping  months  to  the  total  number  of 
months  of  developmental  testing  for  each  program  and  calculated  the 
percentage  of  developmental  testing  done  concurrent  with  production. 

To  examine  programs’  software  development  efforts,  we  identified  the 
programs  that  reported  their  software  as  high-risk.  We  used  the 
questionnaire  responses  from  these  programs  to  assess  the  reasons  why 
they  identified  their  software  effort  as  high-risk.  We  identified  the  dates 
reported  by  programs  for  their  software  and  hardware  integration  and 
compared  those  dates  to  each  program’s  production  start  date  to  assess 
each  programs’  degree  of  software  development  and  production 
concurrency. 


Individual  Assessments  of 
Weapon  Programs 


Of  our  55  total  assessments,  43  are  captured  in  a  two-page  format 
discussing  technology,  design,  and  manufacturing  knowledge  obtained 
and  other  program  issues.  These  two-page  assessments  are  of  major 
defense  acquisition  programs,  most  of  which  are  in  development  or  early 
production.  The  remaining  12  assessments  are  described  in  a  one-page 
format  that  describes  their  current  status.  Those  one-page  assessments 
include  (1)  nine  future  major  defense  acquisition  programs;  (2)  two  major 


This  report  presents  individual  assessments  of  54  current  and  future 
weapon  programs.  In  addition,  we  assessed  the  Navy’s  Frigate  initiative 
separate  from  the  Navy’s  Littoral  Combat  Ship  program,  given  DOD’s 
plans  to  restructure  the  Frigate  into  a  separate  program  soon.  A  table 
listing  these  assessments  is  found  in  appendix  VIII. 
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defense  acquisition  programs  that  are  well  into  production  but  planning  to 
introduce  new  increments  of  capability — the  DDG-51  Flight  III  and  P-8A 
Increment  3  modification  programs;  and  (3)  the  Navy’s  Frigate  initiative. 
For  presentation  purposes  we  grouped  the  individual  assessments  by 
lead  service — Army,  Navy  and  Marine  Corps,  Air  Force,  and  DOD-led — 
and  inserted  a  lead  service  separator  page  at  the  start  of  each  grouping. 
These  four  separator  pages  summarize  information  about  the  acquisition 
phase,  current  estimated  funding  needs,  cost  and  schedule  growth,  and 
product  knowledge  attained  that  is  provided  in  the  one  and  two-page 
assessments.  We  report  cost  and  schedule  growth  in  the  separator  pages 
in  a  manner  that  is  consistent  with  how  it  is  reported  and  described 
elsewhere  in  the  report.  Estimates  of  funding  needed  to  complete  in  the 
separator  pages  are  based  on  all  amounts  that  will  be  provided  in  fiscal 
year  201 7  and  later.  For  some  future  major  defense  acquisition  programs, 
the  estimates  of  funding  needed  represents  only  those  amounts  provided 
through  fiscal  year  2021  and  are  not  the  full  amount  needed  to  complete 
the  acquisition. 

Over  the  past  several  years,  DOD  has  revised  policies  governing  weapon 
system  acquisitions  and  changed  the  terminology  used  for  major 
acquisition  events.  To  make  DOD’s  acquisition  terminology  more 
consistent  across  our  individual  program  assessments,  we  standardized 
the  terminology  for  key  program  events.  For  most  individual  programs  in 
our  assessment,  “development  start”  refers  to  the  initiation  of  an 
acquisition  program  as  well  as  the  start  of  engineering  and  manufacturing 
development  or  system  development.  This  generally  coincides  with 
DOD’s  milestone  B.  A  few  programs  in  our  assessment  have  a  separate 
“program  start”  date,  which  begins  a  pre-system  development  phase  for 
program  definition  and  risk-reduction  activities.  This  “program  start”  date 
generally  coincides  with  DOD’s  former  terminology  for  milestone  I  or 
DOD’s  current  milestone  A,  which  denotes  the  start  of  technology 
maturation  and  risk  reduction.  The  “production  decision”  generally  refers 
to  the  decision  to  enter  the  production  and  deployment  phase,  typically 
with  low-rate  initial  production.  The  “initial  capability”  refers  to  the  initial 
operational  capability — sometimes  called  first  unit  equipped  or  required 
asset  availability.  For  shipbuilding  programs,  the  schedule  of  key  program 
events  in  relation  to  acquisition  milestones  varies  for  each  program.  Our 
work  on  shipbuilding  best  practices  has  identified  the  detail  design 
contract  award  and  the  start  of  lead  ship  fabrication  as  the  points  in  the 
acquisition  process  roughly  equivalent  to  development  start  and  design 
review  for  other  programs. 
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For  each  program  we  assessed  in  a  two-page  format,  we  present  cost, 
schedule,  and  quantity  data  at  the  program’s  first  full  estimate,  and  an 
estimate  from  the  latest  SAR  or  the  program  office  reflecting  2016  data 
where  they  were  available.  The  first  full  estimate  is  generally  the  cost 
estimate  established  at  milestone  B — development  start;  however,  for  a 
few  programs  that  did  not  have  such  an  estimate,  we  used  the  estimate  at 
milestone  C — production  start — instead.  For  shipbuilding  programs,  we 
used  their  planning  estimates  if  those  estimates  were  available.  For 
systems  for  which  a  first  full  estimate  was  not  available,  we  only  present 
the  latest  available  estimate  of  cost  and  quantities.  For  the  other 
programs  assessed  in  a  one-page  format,  we  present  the  latest  available 
estimate  of  cost  and  quantity  from  the  program  office. 

For  each  program  we  assessed,  all  cost  information  is  presented  in  fiscal 
year  2017  dollars.  We  converted  cost  information  to  fiscal  year  2017 
dollars  using  conversion  factors  from  the  DOD  Comptroller’s  National 
Defense  Budget  Estimates  for  Fiscal  Year  2017  (table  5-9).  We  have 
depicted  only  the  program’s  main  elements  of  acquisition  cost — research 
and  development  and  procurement.  Flowever,  the  total  program  cost  also 
includes  military  construction  and  acquisition-related  operation  and 
maintenance  costs.  Because  of  rounding  and  these  additional  costs,  in 
some  situations,  total  cost  may  not  match  the  exact  sum  of  the  research 
and  development  and  procurement  costs.  The  program  unit  costs  are 
calculated  by  dividing  the  total  program  cost  by  the  total  quantities 
planned.  These  costs  are  often  referred  to  as  program  acquisition  unit 
costs.  In  some  instances,  the  data  were  not  applicable,  and  we  annotate 
this  by  using  the  term  “not  applicable  (NA).”  The  quantities  listed  refer  to 
total  quantities,  including  both  procurement  and  development  quantities. 

The  schedule  assessment  for  each  program  is  based  on  acquisition  cycle 
time,  defined  as  the  number  of  months  between  program  start  and  the 
achievement  of  initial  operational  capability  or  an  equivalent  fielding  date. 
In  some  instances  the  data  were  not  yet  available,  and  we  annotate  this 
by  using  the  term  “to  be  determined  (TBD)”  or  “NA.” 

The  information  presented  on  the  “funding  needed  to  complete”  is  from 
fiscal  year  2017  through  completion  and,  unless  otherwise  noted,  draws 
on  information  from  SARs  or  on  data  from  the  program  office.  In  some 
instances,  the  data  were  not  available,  and  we  annotate  this  by  the  term 
“TBD”  or  “NA.”  The  quantities  listed  refer  only  to  procurement  quantities. 
Satellite  programs,  in  particular,  produce  a  large  percentage  of  their  total 
operational  units  as  development  quantities,  which  are  not  included  in  the 
quantity  figure. 
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The  intent  of  these  comparisons  is  to  provide  an  aggregate,  or  overall, 
picture  of  a  program’s  history.  These  assessments  represent  the  sum  of 
the  federal  government’s  actions  on  a  program,  not  just  those  of  the 
program  manager  and  the  contractor.  DOD  does  a  number  of  detailed 
analyses  of  changes  that  attempt  to  link  specific  changes  with  triggering 
events  or  causes.  Our  analysis  does  not  attempt  to  make  such  detailed 
distinctions. 

In  this  year’s  assessment,  we  also  reviewed  whether  individual 
subcontracting  reports  from  a  program’s  prime  contractor  or  contractors 
were  accepted  on  the  Electronic  Subcontracting  Reporting  System 
(eSRS).  We  reviewed  this  information  for  78  of  the  major  defense 
acquisition  programs  included  in  our  assessment  using  the  contract 
information  reported  in  their  December  2015  Selected  Acquisition 
Reports.  See  appendix  VI  for  a  list  of  the  programs  we  reviewed. 


Product  Knowledge  Data 
on  Individual  Two-Page 
Assessments 


In  our  past  work  examining  weapon  acquisition  issues  and  best  practices 
for  product  development,  we  have  found  that  leading  commercial  firms 
pursue  an  acquisition  approach  that  is  anchored  in  knowledge,  whereby 
high  levels  of  product  knowledge  are  demonstrated  by  critical  points  in 
the  acquisition  process.  On  the  basis  of  this  work,  we  have  identified 
three  key  knowledge  points  during  the  acquisition  cycle — system 
development  start,  critical  design  review,  and  production  start — at  which 
programs  need  to  demonstrate  critical  levels  of  knowledge  to  proceed.  To 
assess  the  product  development  knowledge  of  each  program  at  these  key 
points,  we  reviewed  data-collection  instruments  and  questionnaires 
submitted  by  programs;  however,  not  every  program  had  responses  to 
each  element  of  the  data-collection  instrument  or  questionnaire.  We  also 
reviewed  pertinent  program  documentation  and  discussed  the  information 
presented  on  the  data-collection  instrument  and  questionnaire  with 
program  officials  as  necessary. 


In  this  year’s  report  we  have  made  a  change  to  our  attainment  of  product 
knowledge  graphic.  In  addition  to  assessing  programs’  current  status  in 
achieving  the  product  knowledge  criteria,  we  now  also  show  programs’ 
progress  in  meeting  the  knowledge  attainment  criteria  at  the  time  they 
reached  the  three  key  knowledge  points  during  the  acquisition  cycle — 
system  development  start,  critical  design  review,  and  production  start.  For 
programs  that  have  passed  a  key  decision  point  and  have  since  been 
restructured,  we  continue  to  assess  them  against  their  original  cost  and 
schedule  estimates  at  that  milestone  or  decision  point,  such  as 
development  start.  We  have  not  reassessed  a  program  at  milestones  that 


Page  180 


GAO-1 7-333SP  Assessments  of  Selected  Weapon  Programs 


Appendix  I:  Scope  and  Methodology 


have  already  been  reached  if  a  program  is  repeating  a  key  decision  point 
or  milestone,  such  as  milestone  B.  We  have  kept  our  original  assessment 
of  the  program’s  knowledge  attained  at  the  original  milestone.  However, 
we  have  changed  future  milestone  dates  in  instances  when  the  program 
had  not  yet  reached  the  affected  milestone.  In  these  instances,  we 
assessed  the  program  for  its  implementation  of  our  best  practices  at  that 
point  in  time. 

To  assess  a  program’s  readiness  to  enter  system  development,  we 
collected  data  through  the  data-collection  instrument  on  critical 
technologies  and  early  design  reviews.  To  assess  technology  maturity, 
we  asked  program  officials  to  apply  a  tool,  referred  to  as  technology 
readiness  levels  (TRL),  for  our  analysis.  The  National  Aeronautics  and 
Space  Administration  originally  developed  TRLs,  and  the  Army  and  Air 
Force  science  and  technology  research  organizations  use  them  to 
determine  when  technologies  are  ready  to  be  handed  off  from  science 
and  technology  managers  to  product  developers.  TRLs  are  measured  on 
a  scale  from  1  to  9,  beginning  with  paper  studies  of  a  technology’s 
feasibility  and  culminating  with  a  technology  fully  integrated  into  a 
completed  product.  See  appendix  V  for  TRL  definitions.  Our  best- 
practices  work  has  shown  that  a  TRL  7 — demonstration  of  a  technology  in 
its  final  form,  fit,  and  function  within  a  realistic  environment — is  the  level  of 
technology  maturity  that  constitutes  a  low  risk  for  starting  a  product 
development  program.5  For  shipbuilding  programs,  we  have 
recommended  that  this  level  of  maturity  be  achieved  by  the  contract 
award  for  detail  design.6  In  our  assessment,  the  technologies  that  have 
reached  TRL  7,  a  prototype  demonstrated  in  an  operational  environment, 
are  referred  to  as  mature  or  fully  mature.  Those  technologies  that  have 
reached  TRL  6,  a  prototype  very  close  to  final  form,  fit,  and  function 
demonstrated  within  a  relevant  environment,  are  referred  to  as 
approaching  or  nearing  maturity.  Satellite  technologies  that  have 
achieved  TRL  6  are  assessed  as  fully  mature  due  to  the  difficulty  of 
demonstrating  maturity  in  an  operational  environment — space.  In 
addition,  we  asked  program  officials  to  provide  the  date  of  the  system- 


5GAO,  Best  Practices:  Better  Matching  of  Needs  and  Resources  Will  Lead  to  Better 
Weapon  System  Outcomes ,  GAO-01-288  (Washington,  D.C.:  Mar.  8,  2001);  and  Best 
Practices:  Better  Management  of  Technology  Development  Can  Improve  Weapon  System 
Outcomes ,  GAO/NSIAD-99-162  (Washington,  D.C.:  July  30,  1999). 

6GAO,  Best  Practices:  High  Levels  of  Knowledge  at  Key  Points  Differentiate  Commercial 
Shipbuilding  from  Navy  Shipbuilding,  GAO-09-322  (Washington,  D.C.:  May  13,  2009). 
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level  preliminary  design  review.  We  compared  this  date  to  the  system 
development  start  date. 

In  most  cases,  we  did  not  validate  the  program  offices’  selection  of  critical 
technologies  or  the  determination  of  the  demonstrated  level  of  maturity. 
We  sought  to  clarify  the  TRLs  in  those  cases  where  information  existed 
that  raised  concerns.  If  we  were  to  conduct  a  detailed  review,  we  might 
adjust  the  critical  technologies  assessed,  their  readiness  levels 
demonstrated,  or  both.  It  was  not  always  possible  to  reconstruct  the 
technological  maturity  of  a  weapon  system  at  key  decision  points  after  the 
passage  of  many  years.  Where  practicable,  we  compared  technology 
assessments  provided  by  the  program  office  to  assessments  conducted 
by  officials  from  the  Office  of  the  Assistant  Secretary  of  Defense  for 
Research  and  Engineering. 

To  assess  design  stability,  we  asked  program  officials  to  provide  the 
percentage  of  design  drawings  completed  or  projected  for  completion  by 
the  design  review,  the  production  decision,  and  as  of  our  current 
assessment  in  the  data-collection  instrument.  In  most  cases,  we  did  not 
verify  or  validate  the  percentage  of  engineering  drawings  provided  by  the 
program  office.  We  clarified  the  percentage  of  drawings  completed  in 
those  cases  where  information  that  raised  concerns  existed.  Completed 
drawings  were  defined  as  the  number  of  drawings  released  or  deemed 
releasable  to  manufacturing  that  can  be  considered  the  “build  to” 
drawings.  For  shipbuilding  programs,  we  asked  program  officials  to 
provide  the  percentage  of  the  three-dimensional  product  model  that  had 
been  completed  by  the  start  of  lead  ship  fabrication,  and  as  of  our  current 
assessment.  To  gain  greater  insights  into  design  stability,  we  also  asked 
program  officials  to  provide  the  date  they  planned  to  first  integrate  and 
test  all  key  subsystems  and  components  into  a  system-level  integrated 
prototype.  We  compared  this  date  to  the  date  of  the  design  review.  We 
did  not  assess  whether  shipbuilding  programs  had  completed  integrated 
prototypes. 

To  assess  production  maturity,  we  asked  program  officials  for  their 
Manufacturing  Readiness  Level  (MRL)  for  process  capability  and  control 
or  to  identify  the  number  of  critical  manufacturing  processes  and,  where 
available,  to  quantify  the  extent  of  statistical  control  achieved  for  those 
processes  as  a  part  of  our  data-collection  instrument.  In  most  cases,  we 
did  not  verify  or  validate  the  information  provided  by  the  program  office. 
We  clarified  the  number  of  critical  manufacturing  processes  and  the 
percentage  of  statistical  process  control  where  information  existed  that 
raised  concerns.  We  used  a  standard  called  the  Process  Capability  Index, 
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a  process-performance  measurement  that  quantifies  how  closely  a 
process  is  running  to  its  specification  limits.  The  index  can  be  translated 
into  an  expected  product  defect  rate,  and  we  have  found  it  to  be  a  best 
practice.  We  also  used  data  provided  by  the  program  offices  on  their  MRL 
for  process  capability  and  control,  a  sub-thread  tracked  as  part  of  the 
manufacturing  readiness  assessment  process  recommended  by  DOD,  to 
determine  production  maturity.  We  assessed  programs  as  having  mature 
manufacturing  processes  if  they  reported  an  MRL  9  for  that  sub-thread — 
meaning,  that  manufacturing  processes  are  stable,  adequately  controlled, 
and  capable.  To  gain  further  insights  into  production  maturity,  we  asked 
program  officials  whether  the  program  planned  to  demonstrate  critical 
manufacturing  processes  on  a  pilot  production  line  before  beginning  low- 
rate  production.  We  also  asked  programs  on  what  date  they  planned  to 
begin  system-level  developmental  testing  of  a  fully  configured, 
production-representative  prototype  in  its  intended  environment.  We 
compared  this  date  to  the  production  start  date.  We  did  not  assess 
production  maturity  for  shipbuilding  programs. 

Although  the  knowledge  points  provide  excellent  indicators  of  potential 
risks,  by  themselves  they  do  not  cover  all  elements  of  risk  that  a  program 
encounters  during  development,  such  as  funding  instability.  Our  detailed 
reviews  on  individual  systems  normally  provide  a  more  comprehensive 
assessment  of  risk  elements. 

We  conducted  this  performance  audit  from  April  2016  to  March  2017  in 
accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 
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Table  6  contains  the  current  and  first  full  total  acquisition  cost  estimates 
(in  fiscal  year  2017  dollars)  for  each  program  or  element  in  the 
Department  of  Defense’s  (DOD)  2016  major  defense  acquisition  program 
portfolio.  For  each  program  we  show  the  percent  change  in  total 
acquisition  cost  from  the  first  full  estimate,  as  well  as  over  the  past  year 
and  5  years. 


Table  6:  Current  Cost  Estimates  and  First  Full  Estimates  for  DOD’s  2016  Portfolio  of  Major  Defense  Acquisition  Programs 


Fiscal  year  2017  dollars  (in  millions) 


Program  Name 

Current 

total 

acquisition 

cost 

First  Full 
Estimate 
total 
acquisition 
cost 

Change  in 
total 
acquisition 
cost  from  first 
full  estimate 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past  year 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past  5 
years 
(percent) 

Advanced  Extremely  High  Frequency  (AEHF) 
Satellite 

$15,046 

$6,910 

117.7% 

0.6% 

-1 .5% 

AGM-88E  Advanced  Anti-Radiation  Guided 

Missile  (AGM-88E  AARGM) 

2,731 

1,736 

57.3 

18.8 

32.4 

AH-64E  Apache  New  Build  (AH-64E  New  Build) 

2,264 

2,570 

-11.9 

-3.4 

-5.8 

AH-64E  Apache  Remanufacture  (AH-64E 
Remanufacture) 

14,104 

7,855 

79.6 

-0.3 

21.1 

AIM-9X  Block  II  Sidewinder  (AIM-9X  Blk  II) 

3,821 

4,333 

-11.8 

7.8 

-11.8 

Air  and  Missile  Defense  Radar  (AMDR) 

5,426 

6,064 

-10.5 

2.6 

-10.5 

Airborne  and  Maritime/Fixed  Station  (AMF) 

3,314 

8,845 

-62.5 

-8.7 

-62.8 

Armored  Multi-Purpose  Vehicle  (AMPV) 

1 1 ,074 

11,049 

0.2 

0.2 

0.2 

AIM-120  Advanced  Medium  Range  Air-to-Air 
Missile  (AMRAAM) 

24,687 

11,854 

108.3 

-2.6 

-3.5 

Airborne  Warning  and  Control  System  Block 

40/45  Upgrade  (AWACS  Blk  40/45  Upgrade) 

2,849 

3,029 

-5.9 

-0.7 

-5.9 

B-2  Extremely  High  Frequency  SATCOM  and 
Computer  Increment  1  (B-2  EHF  Incl ) 

594 

770 

-22.8 

-0.1 

-12.3 

B61  Mod  12  Lite  Extension  Program  Tailkit 
Assembly  (B61  Mod  12  LEP  TKA) 

1,281 

1,418 

-9.6 

-8.0 

-9.7 

C-130J  Hercules  Transport  Aircraft  (C-130J) 

17,315 

1,029 

1582.0 

0.4 

3.7 

C-5  Reliability  Enhancement  and  Re-engining 
Program  (C-5  RERP) 

7,636 

11,828 

-35.5 

-0.2 

-5.4 

Cooperative  Engagement  Capability  (CEC) 

6,201 

3,193 

94.2 

6.2 

9.8 

CH-47F  Improved  Cargo  Helicopter  (CH-47F) 

16,080 

3,492 

360.4 

-0.8 

2.4 

CH-53K  Heavy  Lift  Replacement  Helicopter 
(CH-53K) 

26,252 

17,958 

46.2 

1.0 

7.9 
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Fiscal  year  2017  dollars  (in  millions) 


Program  Name 

Current 

total 

acquisition 

cost 

First  Full 
Estimate 
total 
acquisition 
cost 

Change  in 
total 
acquisition 
cost  from  first 
full  estimate 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past  year 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past  5 
years 
(percent) 

Chemical  Demilitarization-Assembled  Chemical 
Weapons  Alternatives  (Chem  Demil-ACWA) 

11,226 

2,866 

291.7 

0.3 

1.7 

Combat  Rescue  Helicopter  (CRH) 

8,532 

8,427 

1.2 

0.2 

1.2 

Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier 
(CVN  78) 

37,540 

38,587 

-2.7 

1.9 

1.8 

DDG  1000  Zumwalt  Class  Destroyer  (DDG 

1000) 

23,646 

37,751 

-37.4 

1.4 

3.9 

DDG  51  Arleigh  Burke  Class  Guided  Missile 
Destroyer  (DDG  51) 

122,689 

16,474 

644.8 

5.4 

11.1 

E-2D  Advanced  Hawkeye  Aircraft  (E-2D  AHE) 

21,939 

16,001 

37.1 

0.8 

14.0 

EA-18G  Growler  Aircraft  (EA-18G) 

16,756 

9,736 

72.1 

8.3 

35.4 

Evolved  Expendable  Launch  Vehicle  (EELV) 

59,638 

18,848 

216.4 

-2.5 

61.4 

Enhanced  Polar  System  (EPS) 

1,438 

1,447 

-0.6 

-0.2 

-0.6 

Excalibur  Precision  155mm  Projectiles 
(Excalibur) 

2,045 

5,181 

-60.5 

3.1 

4.0 

F-22  Increment  3.2B  Modernization  (F-22  Inc 

3.2B  Mod) 

1,587 

1,626 

-2.4 

0.3 

-2.4 

F-35  Lightning  II  Program  (F-35) 

336,152 

231,806 

45.0 

-2.2 

-5.1 

Family  of  Advanced  Beyond  Line-of-Sight 
Terminals  (FAB-T) 

4,519 

3,459 

30.7 

3.8 

-7.5 

MQ-8  (Fire  Scout) 

2,935 

2,837 

3.5 

-0.5 

3.5 

Ground/Air  Task  Oriented  Radar  (G/ATOR) 

2,818 

1,590 

77.3 

0.4 

77.3 

Global  Broadcast  Service  (GBS) 

1,341 

625 

114.6 

2.7 

7.3 

Guided  Multiple  Launch  Rocket  System/Guided 
Multiple  Launch  Rocket  Sys  Alt  Warhead 
(GMLRS/GMLRS  AW) 

6,843 

1,918 

256.8 

-1.4 

8.3 

Global  Positioning  System  III  (GPS  III) 

5,772 

4,275 

35.0 

16.1 

22.8 

Next  Generation  Operational  Control  System 
(GPS  OCX) 

4,353 

3,592 

21.2 

15.2 

21.2 

H-1  Upgrades  (4BW/4BN)  (H-1  Upgrades) 

13,241 

3,934 

236.6 

-0.9 

-2.8 

HC/MC-130  Recapitalization  Aircraft  (HC/MC- 
130  Recap) 

14,343 

9,072 

58.1 

-1.6 

1.0 

Integrated  Air  and  Missile  Defense  (IAMD) 

6,501 

5,454 

19.2 

0.9 

8.4 

Intercontinental  Ballistic  Missile  Fuze 

Modernization  (ICBM  Fuze  Mod) 

1,908 

1,890 

1.0 

0.3 

1.0 
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Fiscal  year  2017  dollars  (in  millions) 


Program  Name 

Current 

total 

acquisition 

cost 

First  Full 
Estimate 
total 
acquisition 
cost 

Change  in 
total 
acquisition 
cost  from  first 
full  estimate 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past  year 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past  5 
years 
(percent) 

Integrated  Defensive  Electronic 

Countermeasures  (IDECM)  Sum 

3,031 

2,362 

28.4 

11.9 

17.5 

IDECM  Block  4 

1,195 

753 

58.7 

28.0 

34.1 

IDECM  Blocks  2/3 

1,836 

1,608 

14.2 

3.4 

8.7 

Joint  Air-to-Ground  Missile  (JAGM) 

5,854 

5,841 

0.2 

0.2 

0.2 

Joint  Air-to-Surface  Standoff  Missile  (JASSM) 

7,486 

4,960 

50.9 

0.8 

-8.1 

JASSM  Baseline 

3,363 

2,514 

33.8 

1.5 

-18.0 

JASSM  Extended  Range  (JASSM-ER) 

4,123 

2,446 

68.6 

0.2 

1.9 

Joint  Direct  Attack  Munition  (JDAM) 

9,998 

3,709 

169.6 

7.4 

40.1 

Joint  Light  Tactical  Vehicle  (JLTV) 

20,467 

24,445 

-16.3 

-15.5 

-16.3 

Joint  Precision  Approach  and  Landing  System 
Increment  1A  (JPALS  Inc  1A) 

1,965 

1,098 

79.0 

22.2 

84.3 

Joint  Tactical  Radio  System  Handheld, 

Manpack,  and  Small  Form  Fit  Radios  (JTRS 

HMS) 

9,564 

10,888 

-12.2 

3.6 

63.8 

KC-130J  Transport  Aircraft  (KC-130J) 

9,675 

10,288 

-6.0 

-1.7 

-5.0 

KC-46A  Tanker  Modernization  Proqram  (KC- 
46A) 

43,627 

47,535 

-8.2 

-0.9 

-8.2 

Littoral  Combat  Ship  (LCS) 

26,650 

2,434 

994.8 

25.8 

-25.3 

Littoral  Combat  Ship  -  Mission  Modules  (LCS 
Packages) 

7,101 

7,108 

-0.1 

1.4 

-0.1 

LHA  6  America  Class  Amphibious  Assault  Ship 
(LHA  6) 

9,989 

3,449 

189.6 

-0.9 

-8.8 

LPD  17  San  Antonio  Class  Amphibious 

T ransport  Dock  (LPD  1 7) 

21,502 

12,705 

69.2 

-0.1 

6.1 

M109A7  Family  of  Vehicles  (M109A7  FOV) 

7,405 

7,254 

2.1 

1.7 

2.1 

MH-60R  Multi-Mission  Helicopter  (MH-60R) 

14,670 

6,004 

144.3 

-0.4 

-8.0 

Multifunctional  Information  Distribution  System 
(MIDS) 

5,019 

1,414 

254.8 

21.1 

53.0 

MQ-1C  Gray  Eagle  Unmanned  Aircraft  System 
(MQ-1C  Gray  Eagle) 

5,377 

1,101 

388.2 

1.3 

-3.9 

MQ-4C  Triton  Unmanned  Aircraft  System  (MQ- 
4C  Triton) 

13,123 

13,936 

-5.8 

1.3 

-7.3 

MQ-9  Reaper  Unmanned  Aircraft  System  (MQ-9 
Reaper) 

12,018 

2,860 

320.2 

-2.2 

-6.8 

Mobile  User  Objective  System  (MUOS) 

7,403 

7,291 

1.5 

-5.1 

-2.2 
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Fiscal  year  2017  dollars  (in  millions) 


Program  Name 

Current 

total 

acquisition 

cost 

First  Full 
Estimate 
total 
acquisition 
cost 

Change  in 
total 
acquisition 
cost  from  first 
full  estimate 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past  year 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past  5 
years 
(percent) 

Navy  Multiband  Terminal  (NMT) 

2,314 

2,518 

-8.1 

7.3 

13.4 

P-8A  Poseidon  Multi-Mission  Maritime  Aircraft 
(P-8A) 

33,477 

33,665 

-0.6 

-0.5 

-6.4 

Patriot  Advanced  Capability-3  Missile  Segment 
Enhancement  (PAC-3  MSE) 

6,538 

7,905 

-17.3 

1.8 

-24.1 

Remote  Minehunting  System  (RMS) 

886 

1,564 

-43.3 

-44.3 

-42.2 

Space  Based  Infrared  System  High  (SBIRS 

High) 

19,184 

4,986 

284.7 

-0.8 

-3.6 

Small  Diameter  Bomb  Increment  II  (SDB  II) 

4,296 

5,100 

-15.8 

7.0 

-15.7 

Standard  Missile-6  (SM-6) 

9,195 

6,183 

48.7 

1.5 

34.6 

Space  Fence  Ground-Based  Radar  System 
Increment  1 

1,553 

1,632 

-4.8 

-3.2 

-4.8 

Ship  to  Shore  Connector  Amphibious  Craft 
(SSC) 

4,000 

4,288 

-6.7 

-3.2 

-6.7 

SSN  774  Virginia  Class  Submarine  (SSN  774) 

95,599 

65,574 

45.8 

3.7 

5.2 

Tactical  Tomahawk  RGM-109E/UGM  109E 

Missile  (TACTOM) 

6,987 

2,295 

204.4 

10.5 

-7.2 

Trident  II  (D-5)  Sea-Launched  Ballistic  Missile 

UGM  133A  (Trident  II  Missile) 

58,527 

51,207 

14.3 

0.2 

1.4 

UH-60M  Black  Hawk  Helicopter  (UH-60M  Black 
Hawk) 

26,488 

14,069 

88.3 

-0.3 

-3.0 

V- 22  Osprey  Joint  Services  Advanced  Vertical 

Lift  Aircraft  (V-22) 

63,534 

43,501 

46.1 

1.5 

2.4 

VH-92A  Presidential  Helicopter  Replacement 
Program 

4,819 

4,842 

-0.5 

-1.1 

-0.5 

Wideband  Global  SATCOM  (WGS) 

4,269 

1,295 

229.7 

2.8 

7.1 

Warfighter  Information  Network-Tactical 

Increment  2  (WIN-T  Inc  2) 

10,680 

4,022 

165.5 

1.3 

62.7 

Warfighter  Information  Network-Tactical 

Increment  3  (WIN-T  Inc  3) 

2,038 

17,754 

-88.5 

-0.4 

-86.5 

Source:  GAO  analysis  of  DOD  data.  |  GAO-1 7-333SP 


Notes:  Data  were  obtained  from  DOD’s  Selected  Acquisition  Reports,  acquisition  program  baselines, 
and,  in  some  cases,  program  offices. 
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Portfolio  of  Major  Defense  Acquisition 
Programs  over  5  Years  and  Since  First  Full 
Estimates 


Table  7  shows  the  change  in  research  and  development  cost, 
procurement  cost,  total  acquisition  cost,  and  average  delay  in  delivering 
initial  operational  capability  for  those  programs  in  the  Department  of 
Defense’s  (DOD)  2016  portfolio  over  the  last  5  years  and  since  their  first 
full  cost  and  schedule  estimates. 


Table  7:  Cost  and  Schedule  Changes  for  Programs  in  DOD’s  2016  Portfolio 

Fiscal  year  2017  dollars  (in  billions) 

5  year 
comparison 
(2011-2016) 

Since  first 
full  estimate 
(baseline  to 
2016) 

Change  in  total  research  and  development  cost 

12.2 

100.5 

4.4% 

53.2% 

Change  in  total  procurement  cost 

-5.5 

381.5 

-0.5% 

48.9% 

Change  in  total  other  acquisition  costs3 

1.9 

2.3 

1 6.5% 

21 .3% 

Change  in  total  acquisition  cost 

8.6 

484.3 

0.6% 

49.4% 

Average  delay  in  delivering  initial  capabilities 

1 1 .0  months 

30.8  months 

1 3.4% 

40.0% 

Source:  GAO  analysis  of  DOD  data.  |  GAO-1 7-333SP 

Notes:  Data  were  obtained  from  DOD’s  Selected  Acquisition  Reports  and  acquisition  program 
baselines.  In  a  few  cases  data  were  obtained  directly  from  program  offices.  Some  numbers  may  not 
sum  due  to  rounding. 

aOther  total  acquisition  costs  include  acquisition-related  operation  and  maintenance  and  system- 
specific  military  construction  costs. 
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GAO’s  prior  work  on  best  product-development  practices  found  that 
successful  programs  take  steps  to  gather  knowledge  that  confirm  that 
their  technologies  are  mature,  their  designs  stable,  and  their  production 
processes  are  in  control.  Successful  product  developers  ensure  a  high 
level  of  knowledge  is  achieved  at  key  junctures  in  development.  We 
characterize  these  junctures  as  knowledge  points.  The  Related  GAO 
Products  section  of  this  report  includes  references  to  the  body  of  work 
that  helped  us  identify  these  practices  and  apply  them  as  criteria  in 
weapon  system  reviews.  The  following  table  summarizes  these 
knowledge  points  and  associated  key  practices. 


Table  8:  Best  Practices  for  Knowledge-based  Acquisitions 


Knowledge  Point  1 :  Technologies,  time,  funding,  and  other  resources  match 
customer  needs.  Decision  to  invest  in  product  development 

Demonstrate  technologies  to  a  high  readiness  level — Technology  Readiness  Level  7 — to 
ensure  technologies  will  work  in  an  operational  environment a 

Ensure  that  requirements  for  product  increment  are  informed  by  preliminary  design 
review  using  systems  engineering  process  (such  as  prototyping  of  preliminary  design) 

Establish  cost  and  schedule  estimates  for  product  on  the  basis  of  knowledge  from 
preliminary  design  using  systems  engineering  tools  (such  as  prototyping  of  preliminary 
design) 

Constrain  development  phase  (5  to  6  years  or  less)  for  incremental  development 
Ensure  development  phase  fully  funded  (programmed  in  anticipation  of  milestone) 

Align  program  manager  tenure  to  complete  development  phase 

Contract  strategy  that  separates  system  integration  and  system  demonstration  activities 

Conduct  independent  cost  estimate 

Conduct  independent  program  assessment 

Conduct  major  milestone  decision  review  for  development  start 

Knowledge  Point  2:  Design  is  stable  and  performs  as  expected.  Decision  to  start 
building  and  testing  production-representative  prototypes 

Complete  system  critical  design  review 

Complete  90  percent  of  engineering  design  drawing  packages 

Complete  subsystem  and  system  design  reviews 

Demonstrate  with  system-level  integrated  prototype  that  design  meets  requirements 
Complete  the  failure  modes  and  effects  analysis 
Identify  key  system  characteristics 
Identify  critical  manufacturing  processes 

Establish  reliability  targets  and  growth  plan  on  the  basis  of  demonstrated  reliability  rates 
of  components  and  subsystems 

Conduct  independent  cost  estimate 

Conduct  independent  program  assessment 
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Conduct  major  milestone  decision  review  to  enter  system  demonstration 

Knowledge  Point  3:  Production  meets  cost,  schedule,  and  quality  targets. 
Decision  to  produce  first  units  for  customer 

Demonstrate  manufacturing  processes 

Build  and  test  production-representative  prototypes  to  demonstrate  product  in  intended 
environment 

Test  production-representative  prototypes  to  achieve  reliability  goal 
Collect  statistical  process  control  data 

Demonstrate  that  critical  processes  are  capable  and  in  statistical  control 

Conduct  independent  cost  estimate 

Conduct  independent  program  assessment 

Conduct  major  milestone  decision  review  to  begin  production 


Source:  GAO.  |  GAO-1 7-333SP 

aDOD  considers  Technology  Readiness  Level  6,  demonstrations  in  a  relevant  environment,  to  be 
appropriate  for  programs  entering  system  development;  therefore  we  have  analyzed  programs 
against  this  measure  as  well. 
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Table  9:  Technology  Readiness  Levels  and  Descriptions 

Technology  readiness  level 

Description 

Hardware/software 

Demonstration 

environment 

1 .  Basic  principles  observed 
and  reported 

Lowest  level  of  technology 
readiness.  Scientific  research 
begins  to  be  translated  into  applied 
research  and  development. 
Examples  might  include  paper 
studies  of  a  technology’s  basic 
properties 

None  (paper  studies  and 
analysis) 

None 

2.  Technology  concept  and/or  Invention  begins.  Once  basic  None  (paper  studies  and  None 

application  formulated  principles  are  observed,  practical  analysis) 


applications  can  be  invented.  The 
application  is  speculative  and  there 
is  no  proof  or  detailed  analysis  to 
support  the  assumption.  Examples 
are  still  limited  to  paper  studies. 

3.  Analytical  and  experimental  Active  research  and  development  is  Analytical  studies  and  Lab 

critical  function  and/or  initiated.  This  includes  analytical  demonstration  of  non-scale 

characteristic  proof  of  studies  and  laboratory  studies  to  individual  components  (pieces  of 

concept  physically  validate  analytical  subsystem) 

predictions  of  separate  elements  of 
the  technology.  Examples  include 
components  that  are  not  yet 
integrated  or  representative. 

4.  Component  and/or  Basic  technological  components  are  Low-fidelity  breadboard.  Lab 

breadboard  validation  in  integrated  to  establish  that  the  Integration  of  non-scale 

laboratory  environment  pieces  will  work  together.  This  is  components  to  show  pieces  will 

relatively  “low  fidelity”  compared  to  work  together.  Not  fully 
the  eventual  system.  Examples  functional  or  form  or  fit  but 
include  integration  of  “ad  hoc”  representative  of  technically 

hardware  in  a  laboratory.  feasible  approach  suitable  for 

flight  articles. 


5.  Component  and/or  Fidelity  of  breadboard  technology  High-fidelity  breadboard.  Lab  demonstrating 

breadboard  validation  in  increases  significantly.  The  basic  Functionally  equivalent  but  not  functionality  but  not  form 

relevant  environment  technological  components  are  necessarily  form  and/or  fit  (size  and  fit.  May  include  flight 

integrated  with  reasonably  realistic  weight,  materials,  etc.).  Should  demonstrating 

supporting  elements  so  that  the  be  approaching  appropriate  breadboard  in  surrogate 

technology  can  be  tested  in  a  scale.  May  include  integration  of  aircraft.  Technology 

simulated  environment.  Examples  several  components  with  ready  for  detailed  design 

include  “high  fidelity”  laboratory  reasonably  realistic  support  studies, 

integration  of  components.  elements/subsystems  to 

demonstrate  functionality. 
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Technology  readiness  level  Description  Hardware/software  Demonstration 

environment 

High-fidelity  lab 
demonstration  or 
limited/restricted  flight 
demonstration  for  a 
relevant  environment. 
Integration  of  technology 
is  well  defined. 

a  prototype  in  a  high  fidelity 
laboratory  environment  or  in 
simulated  realistic  environment. 


6.  System/subsystem  model  or 
prototype  demonstration  in  a 
relevant  environment 


Representative  model  or  prototype 
system,  which  is  well  beyond  the 
breadboard  tested  for  TRL  5,  is 
tested  in  a  relevant  environment. 
Represents  a  major  step  up  in  a 
technology’s  demonstrated 
readiness.  Examples  include  testing 


Prototype.  Should  be  very  close 
to  form,  fit  and  function.  Probably 
includes  the  integration  of  many 
new  components  and  realistic 
supporting  elements/subsystems 
if  needed  to  demonstrate  full 
functionality  of  the  subsystem. 


7.  System  prototype  Prototype  near  or  at  planned  Prototype.  Should  be  form,  fit  Flight  demonstration  in 

demonstration  in  an  operational  system.  Represents  a  and  function  integrated  with  representative  realistic 

operational  environment  major  step  up  from  TRL  6,  requiring  other  key  supporting  environment  such  as 

the  demonstration  of  an  actual  elements/subsystems  to  flying  test  bed  or 

system  prototype  in  a  realistic  demonstrate  full  functionality  of  demonstrator  aircraft, 

environment,  such  as  in  an  aircraft,  subsystem.  Technology  is  well 

vehicle  or  space.  Examples  include  substantiated  with  test 

testing  the  prototype  in  a  test  bed  data, 

aircraft. 


8.  Actual  system  completed  Technology  has  been  proven  to  Flight-qualified  hardware  Developmental  Test  and 

and  “flight  qualified”  through  work  in  its  final  form  and  under  Evaluation  in  the  actual 

test  and  demonstration  expected  conditions.  In  almost  all  system  application. 

cases,  this  TRL  represents  the  end 
of  true  system  development. 

Examples  include  developmental 
test  and  evaluation  of  the  system  in 
its  intended  weapon  system  to 
determine  if  it  meets  design 
specifications. 


9.  Actual  system  “flight  proven”  Actual  application  of  the  technology  Actual  system  in  final  form  Operational  Test  and 

through  successful  mission  in  its  final  form  and  under  mission  Evaluation  in  operational 

operations  conditions,  such  as  those  mission  conditions. 

encountered  in  operational  test  and 
evaluation.  In  almost  all  cases,  this 
is  the  end  of  the  last  “bug  fixing” 
aspects  of  true  system 
development.  Examples  include 
using  the  system  under  operational 
mission  conditions. 


Source:  GAO  and  its  analysis  of  National  Aeronautics  and  Space  Administration  data.  |  GAO-1 7-333SP 
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in  the  Electronic  Subcontracting  Reporting 
System 


Table  10  shows  the  number  of  prime  contractors  for  the  programs  we 
assessed  where  an  individual  subcontracting  report  is  reported  as 
acknowledged  during  2016  in  the  Electronic  Subcontracting  Reporting 
System  (eSRS).  We  reviewed  this  information  for  78  major  defense 
acquisition  programs  included  in  our  assessment  that  reported  prime 
contract  information  in  their  December  2015  Selected  Acquisition  Report 
(SAR)  submissions.  The  government  uses  individual  subcontracting 
reports  from  eSRS  as  one  method  of  monitoring  small  business 
participation,  as  this  tool  includes  information  on  contractors’  performance 
against  small  business  subcontracting  goals.  There  are  multiple  reasons 
why  a  prime  contractor  may  not  have  an  acknowledged  subcontracting 
report  in  eSRS.  For  example,  some  contractors  may  have  pending  or 
rejected  reports  within  the  system,  as  all  reports  are  reviewed  prior  to 
acknowledgment.  Not  all  prime  contracts  for  major  defense  acquisition 
programs  are  required  to  submit  individual  subcontracting  reports.  For 
example,  some  contractors  report  small  business  participation  at  a 
corporate  level  as  opposed  to  the  program  level,  and  this  data  is  not 
captured  in  the  individual  subcontracting  reports.1  In  addition,  although  a 
prime  contractor  may  be  required  to  submit  a  report,  it  may  not  yet  have 
done  so  for  the  period  we  reviewed.2 


of  December  2016,  10  major  defense  companies  were  participating  in  the  Test 
Program  for  Negotiation  of  Comprehensive  Small  Business  Subcontracting  Plans  created 
by  the  National  Defense  Authorization  Act  for  Fiscal  Years  1990  and  1991  Pub.  L.  No. 

1 01  -1 89,  §  834  (1989).  These  major  defense  companies  have  each  established  a 
comprehensive  subcontracting  plan  on  a  corporate,  division  or  plant-wide  basis  under 
which  a  single  summary  subcontract  report  is  submitted  semi-annually  for  any  covered 
DOD  contracts.  The  test  program  has  been  extended  by  Congress  several  times  with  the 
current  three  year  extension  made  by  the  Carl  Levin  and  Howard  P.  “Buck”  McKeon 
National  Defense  Authorization  Act  for  Fiscal  Year  2015,  Pub.  L.  No.  113-291  §  821 
(2014)  to  end  on  December  31 , 201 7.  Participation  in  the  test  program  is  on  a  voluntary 
basis  such  that  these  major  defense  companies  may  have  contracts  where  they  are 
reporting  on  an  individual  basis  as  well  as  contracts  where  they  are  reporting  on  a 
comprehensive  basis. 

2For  further  information  on  the  limitations  of  eSRS  and  other  contract  reporting  systems, 
see  GAO,  Federal  Subcontracting:  Linking  Small  Business  Subcontractors  to  Prime 
Contracts  Is  Not  Feasible  Using  Current  Systems,  GAO-1 5-1 1 6  (Washington,  D.C.:  Dec. 
11, 2014). 
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in  the  Electronic  Subcontracting  Reporting 

System 

Table  10:  Major  Defense  Acquisition  Programs’  Individual  Subcontracting  Reports  in  the  Electronic  Subcontracting  Reporting 
System 

Program  name 

Number  of 
contracts  listed 
in  the  December 
2015  SAR 

Contracts  with  an 
accepted  individual 
subcontracting 
report  (as  of 
October  2016) 

Advanced  Extremely  High  Frequency  (AEHF)  Satellite 

2 

0 

AGM-88E  Advanced  Anti-Radiation  Guided  Missile  (AGM-88E  AARGM) 

3 

2 

AH-64E  Apache  New  Build  (AH-64E  New  Build) 

1 

1 

AH-64E  Apache  Remanufacture  (AH-64E  Remanufacture) 

6 

3 

AIM-9X  Block  II  Sidewinder  (AIM-9X  Blk  II) 

6 

0 

Air  and  Missile  Defense  Radar  (AMDR) 

1 

0 

Airborne  and  Maritime/Fixed  Station  (AMF) 

0 

0 

Armored  Multi-Purpose  Vehicle  (AMPV) 

1 

1 

AIM-120  Advanced  Medium  Range  Air-to-Air  Missile  (AMRAAM) 

5 

0 

Airborne  Warning  and  Control  System  Block  40/45  Upgrade  (AWACS  Blk  40/45  Upgrade) 

1 

0 

B-2  Extremely  High  Frequency  SATCOM  and  Computer  Increment  1  (B-2  EHF  Incl) 

1 

1 

B61  Mod  12  Life  Extension  Program  Tailkit  Assembly  (B61  Mod  12  LEP  TKA) 

1 

1 

C-130J  Hercules  Transport  Aircraft  (C-130J) 

9 

1 

C-5  Reliability  Enhancement  and  Re-engining  Program  (C-5  RERP) 

1 

0 

Cooperative  Engagement  Capability  (CEC) 

5 

0 

CH-47F  Improved  Cargo  Helicopter  (CH-47F) 

1 

0 

CH-53K  Heavy  Lift  Replacement  Helicopter  (CH-53K) 

2 

0 

Chemical  Demilitarization-Assembled  Chemical  Weapons  Alternatives  (Chem  Demil- 
ACWA) 

2 

0 

Combat  Rescue  Helicopter  (CRH) 

1 

0 

Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier  (CVN  78) 

4 

3 

DDG  1000  Zumwalt  Class  Destroyer  (DDG  1000) 

2 

2 

DDG  51  Arleigh  Burke  Class  Guided  Missile  Destroyer  (DDG  51) 

5 

5 

E-2D  Advanced  Hawkeye  Aircraft  (E-2D  AHE) 

6 

4 

EA-18G  Growler  Aircraft  (EA-18G) 

6 

3 

Evolved  Expendable  Launch  Vehicle  (EELV) 

4 

2 

Enhanced  Polar  System  (EPS) 

1 

1 

Excalibur  Precision  155mm  Projectiles  (Excalibur) 

0 

0 

F-22  Increment  3.2B  Modernization  (F-22  Inc  3.2B  Mod) 

1 

0 

F-35  Lightning  II  Program  (F-35) 

6 

0 

Family  of  Advanced  Beyond  Line-of-Sight  Terminals  (FAB-T) 

2 

0 

Ground/Air  Task  Oriented  Radar  (G/ATOR) 

2 

0 
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Program  name 

Number  of 
contracts  listed 
in  the  December 
2015  SAR 

Contracts  with  an 
accepted  individual 
subcontracting 
report  (as  of 
October  2016) 

Global  Broadcast  Service  (GBS) 

1 

0 

Guided  Multiple  Launch  Rocket  System/Guided  Multiple  Launch  Rocket  Sys  Alt  Warhead 
(GMLRS/GMLRS  AW) 

5 

0 

Global  Positioning  System  III  (GPS  III) 

2 

0 

H-1  Upgrades  (4BW/4BN)  (H-1  Upgrades) 

4 

3 

HC/MC-130  Recapitalization  Aircraft  (HC/MC-130  Recap) 

2 

0 

Integrated  Air  and  Missile  Defense  (IAMD) 

2 

1 

Intercontinental  Ballistic  Missile  Fuze  Modernization  (ICBM  Fuze  Mod) 

1 

0 

Integrated  Defensive  Electronic  Countermeasures  (IDECM)  Block  4 

4 

2 

Joint  Air-to-Ground  Missile  (JAGM) 

1 

0 

Joint  Air-to-Surface  Standoff  Missile  (JASSM) 

3 

0 

Joint  Direct  Attack  Munition  (JDAM) 

1 

1 

Joint  Light  Tactical  Vehicle  (JLTV) 

1 

1 

Joint  Precision  Approach  and  Landing  System  Increment  1A  (JPALS  Inc  1A) 

1 

0 

Joint  Tactical  Radio  System  Handheld,  Manpack,  and  Small  Form  Fit  Radios  (JTRS  HMS) 

2 

2 

KC-130J  Transport  Aircraft  (KC-130J) 

5 

1 

KC-46A  Tanker  Modernization  Program  (KC-46A) 

4 

3 

Littoral  Combat  Ship  (LCS) 

2 

1 

Littoral  Combat  Ship  -  Mission  Modules  (LCS  Packages) 

0 

0 

LHA  6  America  Class  Amphibious  Assault  Ship  (LHA  6) 

1 

1 

LPD  17  San  Antonio  Class  Amphibious  Transport  Dock  (LPD  17) 

1 

1 

MH-60R  Multi-Mission  Helicopter  (MH-60R) 

4 

0 

Multifunctional  Information  Distribution  System  (MIDS) 

6 

4 

MQ-1C  Gray  Eagle  Unmanned  Aircraft  System  (MQ-1C  Gray  Eagle) 

4 

4 

MQ-4C  Triton  Unmanned  Aircraft  System  (MQ-4C  Triton) 

1 

1 

MQ-8  Fire  Scout 

1 

1 

MQ-9  Reaper  Unmanned  Aircraft  System  (MQ-9  Reaper) 

2 

0 

Mobile  User  Objective  System  (MUOS) 

1 

0 

Navy  Multiband  Terminal  (NMT) 

2 

0 

Next  Generation  Operational  Control  System  (GPS  OCX) 

0 

0 

P-8A  Poseidon  Multi-Mission  Maritime  Aircraft  (P-8A) 

2 

1 

Patriot  Advanced  Capability-3  Missile  Segment  Enhancement  (PAC-3  MSE) 

3 

0 

M109A7  Family  of  Vehicles  (M109A7  FOV) 

0 

0 

Remote  Minehunting  System  (RMS) 

0 

0 

Space  Based  Infrared  System  High  (SBIRS  High) 

4 

0 
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Program  name 

Number  of 
contracts  listed 
in  the  December 
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Contracts  with  an 
accepted  individual 
subcontracting 
report  (as  of 
October  2016) 

Small  Diameter  Bomb  Increment  II  (SDB  II) 

1 

0 

Standard  Missile-6  (SM-6) 

3 

0 

Space  Fence  Ground-Based  Radar  System  Increment  1 

1 

0 

Ship  to  Shore  Connector  Amphibious  Craft  (SSC) 

1 

1 

SSN  774  Virginia  Class  Submarine  (SSN  774) 

2 

2 

Tactical  Tomahawk  RGM-109E/UGM  109E  Missile  (TACTOM) 

2 

0 

Trident  II  (D-5)  Sea-Launched  Ballistic  Missile  UGM  133A  (Trident  II  Missile) 

11 

0 

UH-60M  Black  Hawk  Helicopter  (UH-60M  Black  Hawk) 

1 

0 

V-22  Osprey  Joint  Services  Advanced  Vertical  Lift  Aircraft  (V-22) 

3 

0 

VH-92A  Presidential  Helicopter  Replacement  Program 

1 

0 

Wideband  Global  SATCOM  (WGS) 

1 

1 

Warfighter  Information  Network-Tactical  Increment  2  (WIN-T  Inc  2) 

1 

0 

Warfighter  Information  Network-Tactical  Increment  3  (WIN-T  Inc  3) 

1 

0 

Totals 

190 

62 

Source:  GAO  analysis  of  data  from  DOD  and  eSRS.  |  GAO-1 7-333SP 
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Appendix  VII:  Comments  from  the 
Department  of  Defense 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 

3000  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-3000 


ACQUISITION, 
TECHNOLOGY 
AND  LOGISTICS 


MAR  1  3  2017 


Mr.  Michael  J.  Sullivan 

Director,  Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  GStNW 
Washington,  DC  20548 

Dear  Mr.  Sullivan, 

This  is  the  Department  of  Defense  response  to  the  GAO  Draft  Report,  GAO-17-333SP, 
“DEFENSE  ACQUISITIONS:  Assessments  of  Selected  Weapon  Programs,”  dated 
February  17,  2017  (GAO  Code  100802). 

The  Department  is  pleased  that  this  year’s  Draft  Report,  the  15lh  annual  assessment  on  the 
performance  of  DoD’s  major  acquisition  programs,  identifies  many  ways  in  which  the 
Department  continues  to  drive  down  the  cost  of  the  acquisition  portfolio.  GAO  results 
appear  to  validate  our  focus  on  continuous  improvement  on  the  cost,  schedule,  and 
performance  measures  of  our  programs  using  the  Better  Buying  Power  and  other 
initiatives. 

The  USD(AT&L)  will  again  publish  a  comprehensive  report  on  the  acquisition  system  in 
the  Summer  of  2017.  We  appreciated  the  opportunity  in  previous  years  to  host  question 
and  answer  sessions  regarding  our  report  with  members  of  your  staff,  and  look  forward  to 
continuing  to  share  information  and  best  practices  in  the  future. 

The  Department  appreciates  the  opportunity  to  comment  on  the  Draft  Report.  Technical 
comments  will  be  provided  separately.  My  point  of  contact  for  this  effort  is  CDR  Joseph 
Mitzen,  703-697-8020. 


Performing  the  Duties  of 
Assistant  Secretary  of  Defense  for 
Acquisition 
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Appendix  VIII:  GAO  Contact  and  Staff 
Acknowledgments 


GAO  Contact 

Michael  J.  Sullivan,  (202)  512-4841  or  sullivanm@gao.gov 

o+off  Principal  contributors  to  this  report  were  Christopher  R.  Durbin,  Assistant 

olcl  Director;  Desiree  E.  Cunningham;  Matthew  T.  Drerup;  Danny  G.  Owens; 

Acknowledgments  Wendy  P.  Smythe;  Patrick  R.  Tierney;  and  J.  Andrew  Walker.  Other  key 

contributors  included,  Matthew  Ambrose,  Cheryl  K.  Andrew,  Sonja  J. 
Bensen,  David  B.  Best,  Emily  Bond,  Edwin  B.  Booth,  Julie  C.  Hadley, 

Rich  Horiuchi,  J.  Kristopher  Keener,  Julia  Kennon,  Jill  N.  Lacey, 

Katherine  Lenane,  Travis  J.  Masters,  LaTonya  D.  Miller,  Diana  Moldafsky, 
Meghan  C.  Perez,  Beth  Reed  Fritts,  Max  B.  Sawicky,  Ronald  E. 

Schwenn,  William  Shear,  Charlie  Shivers,  Roxanna  T.  Sun,  Jay  Tallon, 
Bruce  H.  Thomas,  Kristin  VanWychen,  and  Alyssa  B.  Weir. 

Table  1 1  lists  the  staff  responsible  for  individual  programs: 


Table  1 1 :  Staff  Responsible  for  Individual  Program  Assessments 

Program  name 

Primary  staff 

Current  programs 

Air  and  Missile  Defense  Radar  (AMDR) 

Sean  D.  Merrill,  LeAnna  M.  Parkey 

Airborne  &  Maritime/Fixed  Station  (AMF) 

R.  Eli  DeVan 

Amphibious  Combat  Vehicle  (ACV) 

Bonita  J.  P.  Oden,  Patrick  R.  Tierney 

Armored  Multi-Purpose  Vehicle  (AMPV) 

Charlie  Shivers,  Marcus  C.  Ferguson,  Andrea  C.  Evans 

B-2  Defensive  Management  System  Modernization  (B-2  DMS-M) 

Matthew  B.  Lea,  Don  M.  Springman 

CH-53K  Heavy  Lift  Replacement  Helicopter 

Robert  K.  Miller,  Victoria  C.  Klepacz 

Combat  Rescue  Helicopter  (CRH) 

Sean  C.  Seales,  Matthew  T.  Drerup 

Common  Infrared  Countermeasure  (CIRCM) 

Danny  G.  Owens,  Wendy  P.  Smythe 

CVN  78  Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier  (CVN  78) 

Jessica  E.  Karnis,  Burns  C.  Eckert,  Marcia  Fernandez 

DDG  51  Flight  III  Destroyer  (DDG  51  Flight  III) 

Laura  M.  Jezewski,  Karen  L.  Cassidy 

DDG  1000  Zumwalt  Class  Destroyer  (DDG  1000) 

Ramzi  N.  Nemo,  Angie  Nichols-Friedman 

Enhanced  Polar  System  (EPS) 

Andrew  Redd,  Bradley  L.  Terry 

Evolved  Expendable  Launch  Vehicle  (EELV) 

Erin  E.  Cohen,  Andrew  Redd 

F-15  Eagle  Passive  Active  Warning  Survivability  System 
(F-15  EPAWSS) 

Matthew  T.  Drerup,  LeAnna  M.  Parkey 

F-22  Increment  3.2B  Modernization  (F-22  Inc  3.2B  Mod) 

Nathaniel  Vaught,  Sean  C.  Seales 

F-35  Lightning  II  Program  (F-35  JSF) 

Jillena  S.  Roberts,  Megan  L.  Setser 

Family  of  Advanced  Beyond  Line-of-Sight  Terminals  (FAB-T) 

Alexandra  Dew  Silva,  Scott  M.  Purdy 

Global  Positioning  System  III  (GPS  III) 

Jonathan  Mulcare,  Erin  E.  Cohen 
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Program  name 

Primary  staff 

Ground/Air  Task  Oriented  Radar  (G/ATOR) 

Joe  E.  Hunter,  Claire  Li,  Jonathan  Munetz 

Indirect  Fire  Protection  Capability  Increment  2  -  Intercept,  Block  1 

Brian  T.  Smith 

Integrated  Air  and  Missile  Defense  (IAMD) 

Julie  C.  Hadley,  Meredith  A.  Kimmett 

Joint  Air-to-Ground  Missile  (JAGM) 

Jessica  M.  Berkholtz,  John  W.  Crawford 

Joint  Light  Tactical  Vehicle  (JLTV) 

Marcus  C.  Ferguson,  Ethan  1.  Levy,  Andrea  C.  Evans 

Joint  Precision  Approach  and  Landing  System  Increment  1A 
(JPALS  Inc  1  A) 

Stephen  V.  Marchesani,  Jennifer  A.  Dougherty 

Joint  Tactical  Radio  System  Handheld,  Manpack,  and  Small  Form 
Fit  Radios  (JTRS  HMS) 

Scott  M.  Purdy,  Jessica  E.  Karnis 

KC-46A  Tanker  Modernization  Program  (KC-46A) 

Katheryn  S.  Hubbell,  Nathaniel  Vaught 

LHA  6  America  Class  Amphibious  Assault  Ship  (LHA  6) 

Alexis  S.  Olson,  Mackenzie  D.  Verniero 

Littoral  Combat  Ship  (LCS) 

Jacob  L.  Beier,  Laurier  R.  Fish 

Littoral  Combat  Ship  -  Mission  Modules  (LCS  Packages) 

Laurier  R.  Fish,  Jacob  L.  Beier 

M109A7  Family  of  Vehicles  (M109A7  FOV) 

Billy  Allbritton,  George  A.  Bustamante,  Jr. 

Military  Global  Positioning  System  User  Equipment  Increment  1 
(MGUEIncI) 

Claire  Buck,  Patrick  Breiding 

MQ-4C  Triton  Unmanned  Aircraft  System  (MQ-4C) 

Matthew  M.  Shaffer,  Don  M.  Springman 

MQ-8  Fire  Scout  Unmanned  Aircraft  System  (MQ-8  Fire  Scout) 

James  S.  Kim,  Justin  M.  Jaynes 

Next  Generation  Jammer  Increment  1  (NGJ  Inc  1) 

Laura  T.  Holliday,  Daniel  J.  Glickstein 

Next  Generation  Operational  Control  System  (OCX) 

Patrick  Breiding,  Claire  Buck 

Offensive  Anti-Surface  Warfare  Increment  1  (OASuW  Inc  1) 

Thomas  P.  Twambly,  Leslie  C.  Ashton 

Columbia  Class  Ballistic  Missile  Submarine  (SSBN)  (formerly 

Ohio  Replacement) 

James  Madar,  Herbert  J.  Bowsher 

P-8A  Increment  3  Upgrade  Program  (P-8A  Inc  3) 

Heather  B.  Miller,  Jocelyn  C.  Yin 

Patriot  Advanced  Capability-3  Missile  Segment  Enhancement 
(PAC-3  MSE) 

Meredith  Allen  Kimmett,  Brian  T.  Smith 

Ship  to  Shore  Connector  Amphibious  Craft  (SSC) 

Teague  A.  Lyons,  Matthew  M.  Shaffer 

Small  Diameter  Bomb  Increment  II  (SDB  II) 

John  W.  Crawford,  Jessica  M.  Berkholtz 

Space  Fence  Ground  Based  Radar  System  Increment  1  (Space 
Fence  Inc  1) 

Laura  D.  Hook,  Mary  C.  Diop 

Three-Dimensional  Expeditionary  Long-Range  Radar  (3DELRR) 

Claire  Li,  Joe  E.  Hunter 

VH-92A  Presidential  Helicopter  (VH-92A) 

Bonita  J.  P.  Oden,  Ramzi  N.  Nemo 

Warfighter  Information  Network-Tactical  Increment  2  (WIN-T  Inc 

2) 

Ryan  Stott,  Guisseli  Reyes  Turnell 

Future  Programs 

Advanced  Pilot  Trainer  (APT) 

Marvin  E.  Bonner,  Meghan  C.  Perez 

Amphibious  Ship  Replacement  (LX(R)) 

Holly  N.  Williams 

Improved  Turbine  Engine  Program  (ITEP) 

Wendy  P.  Smythe,  Danny  G.  Owens 
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Program  name 

Primary  staff 

Joint  Surveillance  Target  Attack  Radar  System  Recapitalization 
(JSTARS  Recap) 

J.  Andrew  Walker,  Sameena  Ismailjee 

Long  Range  Precision  Fires  (LRPF) 

George  A.  Bustamante,  Jr.,  Billy  Allbritton 

MQ-25  Unmanned  Aircraft  System  (MQ-25)  (formerly  UCLASS) 

Robert  P.  Bullock,  Julie  C.  Hadley 

Frigate 

Sean  D.  Merrill,  LeAnna  M.  Parkey 

Presidential  Aircraft  Recapitalization  (PAR) 

LeAnna  M.  Parkey,  Wendell  K.  Hudson 

John  Lewis  Class  Fleet  Replenishment  Oiler  (T-AO  205)(formerly 
T-AO(X)) 

Jocelyn  C.  Yin,  Jenny  Chow 

Weather  System  Follow-on — Microwave  (WSF-M) 

Brenna  Derritt,  Maricela  Cherveny 

Source:  GAO.  |  GAO-1 7-333SP 
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Related  GAO  Products 


High-Risk  Series:  Progress  on  Many  High-Risk  Areas ,  While  Substantial 
Efforts  Needed  on  Others.  GAO-17-317.  (Washington,  D.C.:  Feb.  15, 
2017). 

Littoral  Combat  Ship  and  Frigate:  Slowing  Planned  Frigate  Acquisition 
Would  Enable  Better-Informed  Decisions.  GAO-1 7-279T.  Washington, 
D.C.:  Dec.  8,  2016. 

Weapon  System  Requirements:  Detailed  Systems  Engineering  Prior  to 
Product  Development  Positions  Programs  for  Success.  GAO- 1 7-77. 
Washington,  D.C.:  Nov.  17,  2016. 

Defense  Acquisitions:  Assessments  of  Selected  Weapon  Programs. 
GAO-1 6-329SP.  Washington,  D.C.:  Mar.  31, 2016. 

KC-46  Tanker  Aircraft:  Key  Aerial  Refueling  Capabilities  Should  Be 
Demonstrated  Prior  to  the  Production  Decision.  GAO- 15-308. 
Washington,  D.C.:  Apr.  9,  2015. 

Federal  Subcontracting:  Linking  Small  Business  Subcontractors  to  Prime 
Contracts  Is  Not  Feasible  Using  Current  Systems.  GAO-1 5-1 1 6. 
Washington,  D.C.:  Dec.  11, 2014. 

National  Defense:  Department  of  Defense’s  Waiver  of  Competitive 
Prototyping  Requirement  for  the  Navy’s  Fleet  Replenishment  Oiler 
Program.  GAO-1 5-57R.  Washington,  D.C.:  Oct.  8,  2014. 

Best  Practices:  DOD  Can  Achieve  Better  Outcomes  by  Standardizing  the 
Way  Manufacturing  Risks  Are  Managed.  GAO-1 0-439.  Washington,  D.C.: 
Apr.  22,  2010. 

Best  Practices:  High  Levels  of  Knowledge  at  Key  Points  Differentiate 
Commercial  Shipbuilding  from  Navy  Shipbuilding.  GAO-09-322. 
Washington,  D.C.:  May  13,  2009. 

Defense  Acquisitions:  A  Knowledge-Based  Funding  Approach  Could 
Improve  Major  Weapon  System  Program  Outcomes.  GAO-08-619. 
Washington,  D.C.:  July  2,  2008. 

Defense  Acquisitions:  Stronger  Management  Practices  are  Needed  to 
Improve  DOD’s  Software-Intensive  Weapon  Acquisitions.  GAO-04-393. 
Washington,  D.C.:  Mar.  1, 2004. 
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Best  Practices:  Capturing  Design  and  Manufacturing  Knowledge  Early 
Improves  Acquisition  Outcomes.  GAO-02-701.  Washington,  D.C.:  July  15, 
2002. 

Best  Practices:  Better  Matching  of  Needs  and  Resources  Will  Lead  to 
Better  Weapon  System  Outcomes.  GAO-01 -288.  Washington,  D.C.:  Mar. 
8,  2001. 

Best  Practices:  Better  Management  of  Technology  Development  Can 
Improve  Weapon  System  Outcomes.  GAO/NSIAD-99-162.  Washington, 
D.C.:  July  30,  1999. 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and  investigative 
arm  of  Congress,  exists  to  support  Congress  in  meeting  its  constitutional 
responsibilities  and  to  help  improve  the  performance  and  accountability  of  the 
federal  government  for  the  American  people.  GAO  examines  the  use  of  public 
funds;  evaluates  federal  programs  and  policies;  and  provides  analyses, 
recommendations,  and  other  assistance  to  help  Congress  make  informed 
oversight,  policy,  and  funding  decisions.  GAO’s  commitment  to  good  government 
is  reflected  in  its  core  values  of  accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost  is 
through  GAO’s  website  (http://www.gao.gov).  Each  weekday  afternoon,  GAO 
posts  on  its  website  newly  released  reports,  testimony,  and  correspondence.  To 
have  GAO  e-mail  you  a  list  of  newly  posted  products,  go  to  http://www.gao.gov 
and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of  production  and 
distribution  and  depends  on  the  number  of  pages  in  the  publication  and  whether 
the  publication  is  printed  in  color  or  black  and  white.  Pricing  and  ordering 
information  is  posted  on  GAO’s  website,  http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)  512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card,  MasterCard, 

Visa,  check,  or  money  order.  Call  for  additional  information. 

Connect  with  GAO 

Connect  with  GAO  on  Facebook,  Flickr,  Linkedln,  Twitter,  and  YouTube. 

Subscribe  to  our  RSS  Feeds  or  E-mail  Updates.  Listen  to  our  Podcasts. 

Visit  GAO  on  the  web  at  www.gao.gov  and  read  The  Watchblog. 

To  Report  Fraud, 

Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Website:  http://www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Katherine  Siggerud,  Managing  Director,  siggerudk@gao.gov,  (202)  512-4400, 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  71 25, 
Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngd@gao.gov,  (202)  512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 

Washington,  DC  20548 

Strategic  Planning  and 
External  Liaison 

James-Christian  Blockwood,  Managing  Director,  spel@gao.gov,  (202)  512-4707 
U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7814, 
Washington,  DC  20548 
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